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FOREWORD 

This  document  contains  the  recommended  Quality  Assurance  Program 
Management  Standard  and  Alternatives  as  well  as  support  material  that  outlines 
the  approach  used  in  its  creation.  This  document  was  developed  under  Air 
Force  Contract  #F33615-81-C-5119,  Project  Priority  1105,  entitled  "ICAM 
Conceptual  Design  for  Computer  Integrated  Manufacturing."  This  contract  was 
sponsored  by  the  Computer  Integrated  Manufacturing  Branch,  Manufacturing 
Technology  Division,  Materials  Laboratory,  Air  Force  Wright  Aeronautical 
Laboratories,  Air  Force  Systems  Command,  Wright-Patterson  Air  Force  Base, 

Ohio,  45433.  This  project  was  administered  under  the  technical  direction  of 
Captain  Richard  R.  Preston. 

This  document.  Volume  IV,  Part  6,  of  the  Final  Technical  Report,  contains 
the  Quality  Assurance  Modeling  and  Analysis  Quality  Assurance  Program 
Management  Standard  Recommendations. 

The  results  of  this  project  have  been  achieved  by  a  coalition  of  companies 
organized  and  managed  under  the  leadership  of  the  prime  contractor,  Vought 
Corporation,  with  Mr.  Don  L.  Norwood  providing  primary  overall  contract 
leadership  and  management  responsibility  (TASK  A).  Other  Task:  leaders  were: 

1.  Mr.  Robert  L.  Moraski,  Vought  Corporation,  responsible  for  leadership 
and  management  of  the  Factory  of  the  Future  Conceptual  Framework  Thrust 
(TASK  B) . 

2.  Mr.  Frank  E.  Sullivan,  Northrop  Corporation,  responsible  for  leadership 
and  management  of  the  Integrated  Composites  Center  Conceptual  and 
Preliminary  Designs  (TASK  C  and  TASK  E) . 

3.  Mr.  Robert  H.  Wettach,  General  Electric  Company,  responsible  for 
leadership  and  management  of  the  Quality  Assurance  Modeling  and 
Analysis  Thrust  (TASK  D) . 

This  effort  was  supported  by  a  coalition  team  consisting  of  the  following 
members : 


Mr.  T.  R.  Lee 
Mr.  J.  J.  Page 
Mr.  B.  R.  Shepherd 
Mr.  M.  G.  Stroud 
Mr.  W.  D.  Vinson 
Mrs.  P.  A.  Waldrop 
Mr.  J.  P.  Watkins 


Northrop  Corporation 
Vought  Corporation 
General  Electric  Company 
Vought  Corporation 
General  Electric  Company 
General  Dynamics/Ft.  Worth 
General  Dynamics/Ft.  Worth 


In  addition  to  the  major  coalition  participants,  the  following  companies 
and  organizations  have  served  as  contributing  and  reviewing  participants  for 
this  document: 
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INDUSTRY  MILITARY 

Alvin  M.  Baughman-Consultant  APPRO  DET  27/General  Dynamics,  Ft.  Worth 

Bendiz  Corporation  HQ-Air  Force  Logistics  Command /QAA 

Boeing  Aerospace  Company  HQ-Air  Force  Systems  Command 

Boeing  Commerical  Airplane  Company  Aeronautical  Systems  Division/PMDQ 

Boeing  Military  Airplane  Company  HQ-Air  Force  Systems  Command  Armament 

Cincinnati  Milacron  Division/ENP 

General  Dynamics/Ft.  Worth  HQ-Air  Force  Systems  Command  Arnold 

General  Electric:  Engineering  Development  Center/PKQ 

Aerospace  Electronic  Systems,  Utica  HQ-Air  Force  Systems  Command  Ballistic 
Aircraft  Engine,  Evendale  Missile  Office/AWQ 

Simulation  and  Control  Systems,  HQ-Air  Force  Systems  Command  Contract 

Daytona  Beach  Management  Division/QA 

Ordnance,  Pittsfield  HQ-Air  Force  Systems  Command  Space 

Military  Electronic  Systems,  Division/PDQ 

Syracuse  HQ-Defense  Logistics  Agency  Cameron 

Re-entry  Systems,  Philadelphia  Station/DLA-QR 

Aerospace  Control  Systems  HQ-NASA,  Office  of  Chief  Engineer 

Binghamton  HQ-Naval  Air  Systems  Command 

Grumman  Aerospace  Office  of  the  Secretary  of  Defense, 

Honeywell,  Inc.  Asst,  for  Quality 

Hughes  Office  of  the  Undersecretary  of 

Lockheed  Missiles  &  Space  Company  Defense,  Research  and  Engineering 

Lockheed-Georgia  USA  DARCOM  (DRXQA-Q) 

Martin  Marietta 
McDonnell  Aircraft 
Rockwell  International 
Sikorsky  Aircraft 
TRW,  Inc. 

Texas  Instruments 
United  Technologies  Corp. 

Vought  Corporation 

Note  that  the  number  and  date  in  the  upper  right  corner  of  each  page  of 
this  document  indicates  that  this  Document  has  been  prepared  according  to  the 
ICAM  Configuration  Management  Life  Cycle  Document  Requirements  and  is  a 
designated  configuration  item. 
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SECTION!  1.0 
EXECUTIVE  SUMMARY 


I . 1  Overview 

Landing  men  on  the  moon  more  than  any  other  recent  event 
emphasized  the  growing  importance  of  quality.  There  is  a 
continuing  struggle  to  advance  the  capability  of  quality 
assurance  systems  to  meet  an  ever  increasing  demand. 
Quantum  improvements  are  needed.  Changes  in  the  basic 
philosophy  and  mechanisms  for  assuring  quality  must  be 
made . 

To  meet  this  need.  Task  "D‘’-Quality  Assurance/Quality 
Control,  was  established  as  a  part  of  ICAM  Project  1105, 
Conceptual  Design  for  Computer  Integrated  Manufacturing. 
Project  1105  had  two  other  major  tasks: 

o  Task  "B"  -  Establish  Factory  of  the  Future  framework 
ami 

o  Task  "C" / "E "  -  Establish  Integrated  Composite  Center 
Conceptual  and  Preliminary  Designs 

All  three  tasks  were  integrated,  with  the  work  of  Task 
"D"  becoming  a  part  of  both  Task  "B"  and  Task  ”C". 

L . 2  Approach 

Task  "D”  included  a  review  of  the  current  Quality 
Assurance/Qua L ity  Control  shortcomings  which  were 
identified  and  considered  as  a  basis  for  the  Quality 
Assurance  Program  Management  Standard.  Three  paths  were 
pursued  in  defining  the  requirements  for  the  proposed 
Quality  Assurance  Program  Management  Standard:  First,  a 
thorough  review  of  applicable,  existing  military  and 
other  government  standards  and  specifications,  as  well  as 
industry  QA/QC  publications,  was  accomplished.  This 
background  information  came  primarily  from  the  list  of 
documents  shown  in  Section  2-Referenced  Documents. 
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Second,  the  rusults  of  the  initial  phase  of  this 
contracl-Understand  the  Problem-were  examined  for  those 
elements  in  the  current  environment  (the  "As-Is“  world) 
that  have  standards  ramifications. 

Finally,  in  order  to  gain  an  even  broader  perspective, 
the  coalition  solicited  feedback  from  government  and 
industrial  sources  on  their  current  and  anticipated 
future  experience  with  MIL-Q-9858A  and  other  military 
documents.  Details  of  the  feedback  are  provided  in  the 
Appendix . 

Two  figures  are  presented  to  enhance  an  understanding  of 
the  process : 

o  Figure  1-1  summarizes  our  approach  pictorially,  whil 

o  Figure  1-2  displays  it  in  an  IDEF0  ( ICAM 

Definition  method,  function  modeling  version  zero) 
format . 
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Results 


This  work  resulted  in  a  recommended  Quality  Assurance 
Program  Management  Standard  for  use  in  the  automated 
environment  of  the  Factory-of-the-Future.  The 
recommended  standard  applies  equally  well  to  the  current 
factory  environment  and  is  designed  to  accommodate 
anticipated  evolutionary  changes. 

In  addition  to  developing  a  recommended  Quality  Assurance 
Program  Management  Standard,  two  alternative  approaches 
have  been  included  in  this  report.  They  are: 

o  MIL-STD-10 9  Additions,  and 

o  MIL-Q-9858A  Rewrite 

All  three  approaches  are  presented  in  Section  3.0, 
Recommendations  for  Improved  Quality  Assurance  Program 
Management  Requirements. 

This  work  is  presented  to  fulfill  the  requirements  of 
Task  "D"  -  Quality  Assurance/Quali ty  Control,  sub-task 
4.4.4  -  Establish  Product  Assurance  Program  Standard, 
which  is  a  part  of  ICAM  Project  1105. 
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REFERENCED 

2 . 1  Applicable  Documents 
Speci f i cat ions : 

MIL-Q-9858 
M IL- 1-4 5208 
MIL-S-52779 

S  tandards : 

MIL-STD-1520 

MIL-STD-1535 

Do D -STD-4  80 
MIL-STD-109 

MIL-STD-4  5662 
M1L-STD-781 

MIL-STD-785 

M IL-STD- 10  5 
MIL-STD-72 1 


DOCUMENTS 


Quality  Program  Requirements 

Inspection  System  Requirements 

Software  Quality  Assurance 
Program  Requirements 


(USAF)  Corrective  Action  and 
Disposition  System  for 
Nonconforming  Material 

(USAF)  Supplier  Quality 
Assurance  Program  Requirements 

Configuration  Control 

Quality  Assurance  Terms  and 
De  f in i t ions 

Calibration  System  Requirements 

Reliability  Testing  for 
Development  and  Production 
Contracts 

Reliability  Program  for  Systems 
and  Equipment,  Development  and 
Production 

Sampling  Procedures  and  Tables 

Definition  of  Terms  for 
Reliability  and  Maintainability 
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Specifications:  (cont'd) 
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2.2  Terms  and  Abbreviations 

FIRMWARE  -  Logic  circuits  in  computer  "read-only"  memory 
that  may  be  altered  by  software  programming. 

HARD  COPY  -  A  printed  copy  of  machine  output,  a  typed  copy 
or  a  manually  written  copy  in  readable  form.  For  example, 
reports,  listings,  documents,  and  summaries. 

PRODUCT  ASSURANCE  ( PA)  -  The  planned,  interdisciplinary  and 
systematic  establishment  of  all  quality  assurance,  quality 
control,  reliability  and  maintainability  actions  necessary 
to  provide  adequate  confidence  on  an  independent  basis 
that:  requirements  are  properly  specified,  that  the  design 

will  achieve  established  requirements,  that  adequate  test, 
inspection  and  evaluation  systems  are  established  to  detect 
noncon formance  and  that  the  final  product  will  perform  the 
required  function(s)  for  the  designed  life  cycle. 

QUALITY  -  The  composite  of  all  the  attributes  or 
characteristics  including  performance  of  an  item  or  product. 

QUALITY  ASSURANCE  -  The  planned  and  systematic 
establishment  of  all  actions  (management/engineering) 
necess  try  to  provide  adequate  confidence  that 
nonconformance  prevention  provisions  and  reviews  are 
established  during  the  design  phase  and  performed 
throughout  the  product  manufacturing  and  life  cycle  phases. 

QUALITY  CONTROL  -  The  planned  and  systematic  application  of 
all  actions  (management/technical )  necessary  to  control  raw 
materials  or  products  through  the  use  of  test,  inspect, 
evaluate,  and  control  of  processes. 

QUALITY  ASSURANCE /QUALITY  CONTROL  (QA/QC)  -  Combine  two 
definitions  above. 

QUALITY  ASSURANCE  PLAN  -  Describes  the  methods,  policies, 
and  procedures  necessary  to  conduct  quality  assurance 
activities  during  the  design,  manufacturing  and  delivery 
phases . 
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QUALITY  REQUIREMENTS  -  Quality  standards  on  1)  major 
components  /assemblies  and  final  product  characteristics, 
and  2)  applicable  QA/QC  contract  requirements  not 
direct  iy  related  to  product  application. 

SOFT  COPY  -  Data  presented  as  a  video  image,  in  audio  or 
any  other  form  which  is  not  hard  copy. 

SOFTWARE  -  A  set  of  computer  programs,  procedures,  rules, 
and  possibly  associated  documentation  concerned  with  the 
operation  of  a  data  processing  system,  e.g.,  compilers, 
library  routines,  manuals,  and  circuit  diagrams. 
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SECTION  3.0 

RECOMMENDATIONS  FOR  IMPROVED  QUALITY  ASSURANCE  PROGRAM 
MANAGEMENT  REQUIREMENTS 


3 . 1  Quality  Assurance  Program  Management  Standard 


3.1.1  Objective 


To  establish  a  Quality  Assurance  Program  Management 
Standard  flexible  enough  to  accommodate  both  the 
automated  and  non-automated  environments  without  changing 
the  basic  structure  and  requirements  of  MIL-Q-9858A.  The 
recommended  approach  maintains  consistency  with  current 
practices  while  recognizing  essential  interfaces  with 
evolving  production  systems. 


3.1.2  S  tandard 


The  following  section  contains  the  recommended  Quality 
Assurance  Program  Management  Standard. 


\ 
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RECOMMENDED 

QUALITY  ASSURANCE  PROGRAM  MANAGEMENT  STANDARD 


SCOPE 

Applicability.  This  standard  shall  apply  to  all 
supplies  (including  equipments,  subsystems  and  systems) 
or  services  when  referenced  in  the  item  specification, 
contract  or  order. 

Contractual  Intent.  This  standard  requires  the 
establishment  of  a  quality  program  by  the  contractor  to 
assure  compliance  with  the  requirements  of  the  contract. 
The  program  and  procedures  used  to  implement  this 
specification  shall  be  developed  by  the  contractor.  The 
quality  program,  including  procedures,  processes  and 
product  shall  be  documented  in  hardcopy,  such  as  on 
paper,  transparency  or  film,  or  in  softcopy  such  as  disc, 
cartridge  or  tape  and  shall  be  subject  to  review  by  the 
government  representative.  The  contractor  must  make  the 
documentation  available  to  the  Government  Representative 
in  a  convenient  useable  form.  The  quality  program  is 
subject  to  the  disapproval  of  the  Government 
Representative  whenever  the  contractor’s  procedures  do 
not  accomplish  their  objectives.  The  Government,  at  its 
option,  may  furnish  written  notice  of  the  acceptability 
of  the  contractor's  quality  program. 

Summary.  An  effective  and  economical  quality  program, 
planned  and  developed  in  consonance  with  the  contractor's 
other  administrative  and  technical  programs,  is  required 
by  this  spec i f ica t ion .  Design  of  the  program  shall  be 
based  upon  cons ideration  of  the  technical  and 
manufacturing  aspects  of  production  and  related 
engineering  design  and  materials.  The  program  shall 
assure  adequate  quality  throughout  all  areas  of 
pre-contract,  contract  and  post-contract  performance;  for 
example,  proposal  generation,  design,  development, 
fabrication,  processing,  assembly,  inspection,  test, 
maintenance,  packaging,  shipping,  storage,  site 
installation  and  data  feedback  from  end  users. 
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All  supplies!  and  services  under  the  contract,  whether 
manufactured  or  performed  within  the  contractor's  plant 
or  at  any  other  source,  shall  be  controlled  at  all  points 
necessary  to  assure  conformance  to  contractual 
requirements.  The  program  shall  provide  for  the 
prevention  and  ready  detection  of  discrepancies  and  for 
timely  and  positive  corrective  action.  The  contractor 
shall  make  objective  evidence  of  quality  conformance 
readily  available  to  the  Government  Representative. 
Objective  evidence  may  be  in  form  of  written  material, 
graphs,  charts  and  pictures  either  on  paper,  film  or  as  a 
display  on  a  computer  terminal  or  video  screen. 
Instructions  and  records  for  quality  must  be  controlled. 

The  authority  and  responsibility  of  those  in  charge  of 
the  design,  development,  production,  testing,  and  quality 
activities  shall  be  clearly  stated.  The  program  shall 
facilitate  determinations  of  the  effects  of  quality 
deficiencies  and  quality  costs  on  price.  Facilities  and 
standards  such  as  drawings,  engineering  changes, 
measuring  equipment  and  the' like  which  are  necessary  for 
the  creation  of  the  required  quality  shall  be  effectively 
managed.  The  program  shall  include  an  effective  control 
of  purchased  materials  and  subcontracted  work.  Design, 
development,  manufacturing,  fabrication,  assembly, 
testing  and  inspection  work  conducted  within  the 
contractor's  plant  shall  be  controlled.  The  quality 
program  shall  also  include  effective  implementation  of 
responsibilities  shared  jointly  with  the  Government  or 
related  to  Government  functions,  such  as  control  of 
Government  property  and  Government  source  inspection. 

Relation  to  Other  Contract  Requirements.  This  standard 
and  any  procedure  or  document  executed  in  implementation 
thereof,  shall  be  in  addition  to  and  not  in  derogation  of 
other  contract  requirements.  The  quality  program 
requirements  set  forth  in  this  specification  shall  be 
satisfied  in  addition  to  all  detail  requirements 
contained  in  the  statement  of  work  or  in  other  parts  of 
the  contract.  The  contractor  is  responsible  for 
compliance  with  all  provisions  of  the  contract  and  for 
furnishing  specified  supplies  and  services  which  meet  all 
the  requirements  of  the  contract.  If  any  inconsistency 
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exists  between  the  contract  schedule  or  its  general 
provisions  and  this  specification,  the  contract  schedule 
and  the  general  provisions  shall  control.  The 
contractor's  quality  program  shall  be  planned  and  used  in 
a  manner  to  support  reliability  effectively. 

1.5  Relation  to  MIL-Q-985UA.  This  standard  may  be  used  in 
place  of  MIL-Q-9858A. 

1.6  Tailoring.  In  accordance  with  DOD  principles  governing 
the  application  and  tailoring  of  specifications  and 
standards  to  achieve  cost  effective  acquisitions  and  life 
cycle  ownership  of  defense  material,  this  standard  and 
its  supporting  documents  shall  be  tailored  to  the 
specific  program  and  program  phase  within  the  overall 
life  cycle.  This  tailoring  shall  encompass  the  selection 
and  application  of  methods,  tables,  sections,  individual 
paragraphs  and/or  sentences  to  be  placed  on  contract  in 
order  to  impose  only  the  essential  needs  to  preclude 
unnecessary  and  unreasonable  contract  costs. 

2.  SUPERSEDING,  SUPPLEMENTATION  AND  ORDERING 

2.1  Applicable  Documents.  The  following  documents  of  the 
issue  in  effect  on  date  of  the  solicitation  form  a  part 
of  this  standard  to  the  extent  specified  herein. 

SPECIFICATIONS 

Military 

MIL-I-45208A  --  Inspection  System  Requirements 

MIL-STD-4 5662A  --  Calibration  System  Requirements 
MIL-S-52779A  —  Software  Quality  Assurance  Program 

Requirements 

2.2  Amendments  and  Revisions.  Whenever  this  standard  is 
amended  or  revised  subsequent  to  its  contractually 
effective  date,  the  contractor  may  follow  or  authorize 
his  subcontractors  to  follow  the  amended  or  revised 
document  provided  no  increase  in  price  or  fee  is 
required.  The  contractor  shall  not  be  required  to  follow 
the  amended  or  revised  document  except  as  a  change  in 
contract.  If  the  contractor  elects  to  follow  the  amended 
or  revised  document,  he  shall  notify  the  Contracting 
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Officer,  in  writing,  of  this  election.  When  the 
contractor  elects  to  follow  the  provisions  of  an 
amendment  or  revision,  he  must  follow  them  in  full. 

2.3  Ordering  Government  Documents.  Copies  of 

speci f ica t ions ,  standards  and  drawings  required  by 
contractors  in  connection  with  specific  procurements  may 
be  obtained  from  the  procuring  agency,  or  as  otherwise 
directed  by  the  Contracting  Officer. 

3.  QUALITY  PROGRAM  MANAGEMENT 

3.1  Organization.  Effective  management  for  quality  shall 
be  clearly  prescribed  by  the  contractor.  Personnel 
performing  quality  functions  shall  hc.ve  sufficient, 
well-defined  responsibility,  authority  and  the 
organizational  freedom  to  identify  and  evaluate  quality 
problems  and  to  initiate,  recommend  or  provide 
solutions.  Management  regularly  shall  review  the  status 
and  adequacy  of  the  quality  program.  The  term  "quality 
program  requirements"  as  used  herein  identifies  the 
collective  requirements  of  this  specification.  It  does 
not  mean  that  the  fulfillment  of  the  requirements  of  this 
specification  is  the  responsibility  of  any  single 
contractor's  organization,  function  or  person. 

3.2  Early  Involvement  of  Quality.  The  contractor  shall 
include  as  part  of  his  quality  assurance  program  plan  a 
defined  strategy  for  the  early  integration  of  the 
functions  of  marketing,  design,  manufacturing,  quality 
and  the  customer.  The  stategy  shall  provide  for  a  review 
of  the  customer's  requirements  and  expectations  including 
the  identification  of  the  key  program  elements  that  are 
critical  for  program  success.  The  plan  shall  make  timely 
provisions  for  adequate  processes,  equipment,  tooling, 
personnel  skills  and  program  controls  to  assure  meeting 
the  critical  requirements  and  to  assure  overall  product 
quality.  This  initial  planning  will  recognize  the  need 
and  provide  for  research,  when  necessary,  to  update 
inspection  and  testing  techniques,  instrumentation  and 
correlation  of  inspection  and  test  results  with 
manufacturing  methods  and  processes.  This  planning  will 
also  provide  appropriate  review  and  action  to  assure 
compatibility  of  manufacturing,  inspection,  testing  and 
documentation.  The  early  integration  of  the  functional 
quality  activities  is  essential. 
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3-3  Prevention.  The  quality  assurance  program  plan  shall 
focus  on  preventing  quality  problems.  This  should 
include  early  feedback  of  quality,  reliability  and 
maintainability  data  to  engineering  and  manufacturing  for 
design  or  process  changes. 

3.4  Work  Instructions.  The  quality  program  shall  assure 
that  all  work  affecting  quality  (including  such  things  as 
purchasing,  handling,  machining,  assembling,  fabricating, 
processing,  inspection,  testing,  modification, 
installation,  and  any  other  treatment  of  product, 
facilities,  standards  or  equipment  from  design  and 
ordering  of  materials  to  dispatch  of  shipments)  shall  be 
prescribed  in  clear  and  complete  documented  instructions 
of  a  type  appropriate  to  the  circumstances.  Work 
instructions  may  be  in  hard  copy  form  such  as  written 
instructions  on  paper  or  film,  or  in  soft  copy  form  such 
as  computer  tapes  and  discs.  Work  instructions  may  be 
transmitted  to  an  operator  or  directly  to  a  machine. 

Such  instructions  shall  provide  the  criteria  for 
performing  the  work  functions  and  they  shall  be 
compatible  with  acceptance  criteria  for  workmanship. 

The  instructions  are  intended  also  to  serve  for 
supervising,  inspecting  and  managing  work.  The 
preparation  and  maintenance  of  and  compliance  with  work 
instructions  shall  be  monitored  as  a  function  of  the 
quality  program. 

3.5  Records.  The  contractor  shall  maintain  and  use  any 
records  or  data  essential  to  the  economical  and  effective 
operation  of  his  quality  program.  These  records,  either 
as  hard  copy  or  as  soft  copy  such  as  computer  tapes  or 
discs,  shall  be  available  for  review  by  the  Government 
Representative  and  copies  of  individual  records  shall  be 
furnished  him  upon  request.  The  requirement  of 
furnishing  the  Government  Representative  copies  of 
records  shall  be  satisfied  by  furnishing  a  hard  copy  of 
the  records  or  by  furnishing  a  soft  copy,  such  as  a  tape 
or  disc,  capable  of  being  viewed  at  a  computer  terminal 
or  a  video  screen.  The  requirement  may  also  be  satisfied 
by  providing  the  Government  Representative  access  to  a 
computer  terminal  or  a  video  screen  where  the  information 
can  be  seen.  Records  are  considered  one  of  the  principal 
forms  of  objective  evidence  of  quality.  The  quality 
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program  shall  assure  that  records  are  complete  and 
reliable.  Inspection  and  testing  records  shall,  as  a 
minimum,  indicate  the  nature  of  the  observations  together 
with  the  number  of  observations  made  and  the  number  and 
type  of  deficiencies  found.  Also,  records  for 
monitoring  work  performance  and  for  inspection  and 
testing  shall  indicate  the  acceptability  of  work  or 
products  and  the  action  taken  in  connection  with 
deficiencies.  The  quality  program  shall  provide  for  the 
analysis  and  use  of  records  as  a  basis  for  management 
action. 


3.6 


Software.  The  contractor  shall  establish  and  implemen 
a  Software  Quality  Assurance  Program  for  computer 
controlled  automated  processes  and  other  computer  rela 
processes.  This  Program  shall  be  documented  and  shall 
subject  to  review  and  disapproval  by  the  Government 
Representative  whenever  the  contractor's  procedures  do 
not  accomplish  their  objectives. 


t 

ted 

be 


The  software  Quality  Program  shall  be  an  integral  part  of 
the  total  Quality  Assurance  Program.  For  items  specified 
for  delivery  on  a  contract  MIL-S-52779A  shall  apply. 

This  applies  to  either  software  alone  or  to  software  as  a 
portion  of  a  system  or  a  subsystem.  In  this  regard  the 
term  software  also  includes  firmware. 


For  internal  software  used  for  process  control  and 
production  MIL-S-52779A  may  be  used  with  tailoring  to 
meet  specific  requirements.  At  a  minimum  the  process 
control  Software  Quality  Program  shall  address: 

o  A  Program  plan  establishing  strategy,  authority 
and  control 

o  Documentation  including  procedures  and  work 
ins  truct ion 

o  Design/code  reviews  including  Quality 
participation 

o  Configuration  management  including 

identification,  control  and  records  of  change 

o  Software  security 

o  Validation,  verification  and  certification 

o  Audit  programs 

o  Problem  reporting  and  corrective  action 


i 
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The  Software  Quality  Program  shall  be  reviewed  on  an 
established  periodic  basis  to  assure  applicability  to  the 
most  current  contract  requirements. 

3.7  Corrective  Action.  The  quality  program  shall  detect 
promptly  and  correct  assignable  conditions  adverse  to 
quality.  Design,  purchasing,  manufacturing,  testing  or 
other  operations  which  could  result  in  or  have  resulted 
in  defective  supplies,  services,  facilities,  technical 
data,  standards  or  other  elements  of  contract  performance 
which  could  create  excessive  losses  or  costs  must  be 
identified  and  changed  as  a  result  of  the  quality 
program.  Corrective  action  will  extend  to  the 
performance  of  all  suppliers  and  vendors  and  will  be 
responsive  to  data  and  product  forwarded  from  users. 
Corrective  action  shall  include  as  a  minimum: 

(a)  Analysis  of  data  and  examination  of  product  scrapped 
or  reworked  to  determine  extent  and  causes? 

(b)  Analysis  of  trends  in  processes  or  performance  of 
work  to  prevent  nonconf orming  product?  and 

(c)  Introduction  of  required  improvements  and 
corrections,  an  initial  review  of  the  adequacy  of 
such  measures  and  monitoring  of  the  effectiveness  of 
corrective  action  taken. 

3.8  Costs  Related  to  Quality.  The  contractor  shall 
maintain  and  use  quality  cost  data  as  a  management 
element  of  the  quality  program.  These  data  shall  serve 
the  purpose  of  identifying  the  cost  of  both  the 
prevention  and  correction  of  nonconforming  supplies 
(e.g.,  labor  and  material  involved  in  material  spoilage 
caused  by  defective  work,  correction  of  defective  work 
and  for  quality  control  exercised  by  the  contractor  at 
subcontractor's  or  vendor's  facilities).  The  specific 
quality  cost  data  to  be  maintained  and  used  will  be 
determined  by  the  contractor.  These  data  shall,  on 
request,  be  identified  and  made  available  for  "on  site" 
review  by  the  Government  Representative. 

A  cost-benefit  analysis  will  be  required  as  support  for  a 
request  to  "tailor"  any  part  of  this  specification. 
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FACILITIES  AND  STANDARDS 

Design  Review.  Personnel  from  the  contractor's 
engineering,  manufactr.r ing  and  quality  assurance 
organizations  shall  conduct  a  continuing  review  of  design 
and  technical  documentation  and  changes  thereto,  to 
determine  their  adequacy  regarding  design  and  function  of 
the  product,  that  all  necessary  information  has  been 
included,  and  that  the  requirements  are  clear  and 
unambigious  and  conform  to  standard  engineering 
practices.  The  quality  assurance  participant  shall 
ensure  that  reviews  are  documented,  deficiencies 
reported,  and  corrective  action  has  been  taken  prior  to 
document  release.  These  reviews  shall  be  used  in 
subsequent  quality  planning  for  procurement,  fabrication, 
inspection,  and  test. 

Design  Drawings,  Documentation  and  Changes.  The 
contractor  shall  assure  the  adequacy,  completeness  and 
currentness  of  design  documents  by  establishing  and 
implementing  a  conf igurat ion  management  plan.  Design 
documents  (drawings  and  specifications)  include  drawings 
and  specifications  either  in  hard  copy  form  or  in  soft 
copy  form  with  respect  to  computer  tapes  or  discs.  With 
respect  to  the  currentness  of  design  documents  and 
changes,  the  contractor  shall  assure  that  requirements 
for  the  effectivity  point  of  changes  are  met  and  that 
obsolete  design  documents  and  change  requirements  are 
removed  from  all  points  of  issue  and  use.  Some  means  of 
recording  the  effective  points  shall  be  employed  and  be 
available  to  the  Government. 

With  respect  to  design  documents,  a  procedure  shall  be 
maintained  that  provides  for  the  evaluation  of  the 
engineering  adequacy  of  proposed  changes.  The  evaluation 
shall  encompass  both  the  adequacy  in  relation  to 
state-of-the-art  engineering  and  design  practices  and  the 
adequacy  with  respect  to  the  design  and  purpose  of  the 
product  to  which  the  drawing  relates. 
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With  respect  to  supplemental  specifications,  process 
instructions,  production  engineering  instructions  and 
work  instructions  relating  to  a  particular  design,  the 
contractor  shall  be  responsible  for  a  review  of  their 
adequacy,  currentness  and  completeness .  The  quality 
program  must  provide  complete  coverage  of  all  information 
necessary  to  produce  an  article  in  complete  conformity 
with  requirements  of  the  design. 

The  quality  program  shall  assure  that  there  is  complete 
compliance  with  contract  requirements  for  proposing, 
approving,  and  effecting  of  engineering  changes.  The 
quality  program  shall  provide  for  monitoring  effectively 
compliance  with  contractual  engineering  changes  requiring 
approval  by  Government  design  authority.  The  quality 
program  shall  provide  for  monitoring  effectively  the 
design  documentation  changes  of  lesser  importance  not 
requiring  approval  by  Government  design  authorities. 


Delivery  of  correct  design  documentation  and  change 
information  to  the  Government  in  connection  with  data 
acquisition  shall  be  an  integral  part  of  the  quality 
program.  This  includes  full  compliance  with  contract 
requirements  concerning  rights  and  data  both  proprietary 
and  other.  The  quality  program's  responsibility  for 
drawings  and  changes  extend  to  the  drawings  and  changes 
provided  by  the  subcontractors  and  vendors  for  the 
con  tract . 


4.2  Measuring  and  Testing  Equipment.  The  contractor  shall 
provide  and  maintain  gages  and  other  measuring  and 
testing  devices  necessary  to  assure  that  supplies  conform 
to  technical  requirements.  These  devices  shall  be 
calibrated  against  certified  measurement  standards  which 
have  known  valid  relationships  to  national  standards  at 
established  periods  to  assure  continued  accuracy.  The 
objective  is  to  assure  that  inspection  and  test  equipment 
is  adjusted,  replaced  or  repaired  before  it  becomes 
inaccurate.  The  calibration  of  measuring  and  testing 
equipment  shall  be  in  conformity  with  military 
specification  MIL-STD-4 5662A.  In  addition,  the 
contractor  shall  insure  the  use  of  only  such 
subcon  tractor  and  venuor  sources  that  depend  upon 
calibration  systems  which  effectively  control  the 
accuracy  of  measuring  and  testing  equipment. 
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4.3  Production  Tooling  Used  as  Media  of  Inspection.  When 
production  jigs,  fixtures,  tooling  masters,  templates, 
patterns  and  such  other  devices  are  used  as  media  of 
inspection,  they  shall  be  proved  for  accuracy  prior  to 
release  for  use.  These  devices  shall  be  proved  again  for 
accuracy  at  intervals  formally  established  in  a  manner  to 
cause  their  timely  adjustment,  replacement  or  repair 
prior  to  becoming  inaccurate. 

4.4  Automated  Equipment  and  Processes.  Automated  equipment 
may  be  used  as  the  media  of  inspection  when  the  equipment 
or  process  assures  conformance  to  requirements  by  the  way 
it  is  designed  and  controlled.  Completion  of  the 
automated  operation  may  be  accepted  as  objective  evidence 
of  conformance  to  requirements.  When  automated  equipment 
or  processes  are  used  as  the  measure  of  conformance  they 
shall  be  audited  on  a  timely  basis  to  assure  the  proper 
calibration  and  maintenance. 

4.5  Use  of  Contractor's  Inspection  Equipment.  The 
contractor's  gages,  measuring  and  testing  devices  shall 
be  made  available  for  use  by  the  Government  when  required 
to  determine  conformance  with  contract  requirements.  If 
conditions  warrant,  contractor's  personnel  shall  be  made 
available  for  operation  of  such  devices  and  for 
verification  of  their  accuracy  and  condition. 

4.6  Advanced  Metrology  Requirements.  Tne  quality  program 
shall  include  timely  identification  and  report  to  the 
Contracting  Officer  of  any  precision  measurement  need 
exceeding  the  known  state  of  the  art. 

5.  CONTROL  OF  PURCHASES 

5.1  Responsibility.  The  contractor  is  responsible  for 

assuring  that  all  supplies  and  services  procured  from  his 
suppliers  (subcontractors  and  vendors)  conform  to  the 
contract  requirements.  The  selection  of  sources  and  the 
nature  and  extent  of  control  exercised  by  the  contractor 
shall  be  dependent  upon  the  type  of  supplies,  his 
supplier's  demonstrated  capability  to  perform,  and  the 
quality  evidence  made  available.  To  assure  an  adequate 
and  economical  control  of  such  material,  the  contractor 
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shall  utilize  to  the  fullest  extent  objective  evidence  of 
quality  furnished  by  his  suppliers.  When  the  Government 
elects  to  perform  inspection  at  a  supplier's  plant,  such 
inspection  shall  not  be  used  by  contractors  as  evidence 
of  effective  control  of  quality  by  such  suppliers.  The 
inclusion  of  a  product  on  the  Qualified  Products  List 
only  signifies  that  at  one  time  the  manufacturer  made  a 
product  which  met  specification  requirements.  It  does 
not  relieve  the  contractor  of  his  responsibility  for 
furnishing  supplies  that  meet  all  specification 
requirements  or  for  the  performance  of  specified 
inspections  and  tests  for  such  material.  The 
effectiveness  and  integrity  of  the  control  of  quality  by 
his  suppliers  shall  be  assessed  and  reviewed  by  the 
contractor  at  intervals  consistent  with  the  complexity 
and  quantity  of  product.  Inspection  of  products  upon 
delivery  to  the  contractor  shall  be  used  for  assessment 
and  review  to  the  extent  necessary  for  adequate  assurance 
of  quality.  Test  reports,  inspection  records, 
certificates  and  other  suitable  evidence  relating  to  the 
supplier's  control  of  quality  should  be  used  in  the 
contractor's  assessment  and  review.  The  contractor's 
responsibility  for  the  control  of  purchases  includes  the 
establishment  of  a  procedure  for  (1)  the  selection  of 
qualified  suppliers,  (2)  the  transmission  of  applicable 
design  and  quality  requirements  in  the  Government 
contracts  and  associated  technical  requirements,  (3)  the 
evaluation  of  the  adequacy  of  procured  items,  and  (4) 
effective  provisions  for  early  information  feedback  and 
correction  of  nonconformances. 

5.2  Purchasing  Data.  The  contractor's  quality  program 

shall  not  be  acceptable  to  the  Government  unless  the 
contractor  requires  of  his  subcontractors  a  quality 
effort  achieving  control  of  the  quality  of  the  services 
and  supplies  which  they  provide.  The  contractor  shall 
assure  that  all  applicable  requirements  are  properly 
included  or  referenced  in  all  purchase  orders  for 
products  ultimately  to  apply  on  a  Government  contract. 

The  purchase  order  shall  contain  a  complete  description 
of  the  supplies  ordered  including,  by  statement  or 
reference,  all  applicable  requirements  for 
manufacturing,  inspecting,  testing,  packaging,  and  any 
requirements  for  Government  or  contractor  inspections, 
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qualification  or  approvals.  Toclmic.il  requirements  of 
the  following  nature  must  be  included  by  statement  or 
reference  as  a  part  of  the  required  clear  description: 
all  pertinent  design  documentation,  engineering  design 
change  orders,  specifications  (including  inspection 
system  or  quality  program  requirements),  reliability, 
safety,  weight,  or  other  special  requirements,  unusual 
test  or  inspection  procedures  or  equipment  and  any 
special  revision  or  model  identification.  The 
description  of  products  ordered  shall  include  a 
requirement  for  contractor  inspection  at  the 
subcontractor  or  vendor  source  when  such  action  is 
necessary  to  assure  that  the  contractor's  quality  program 
effectively  implements  the  contractor's  responsibility 
for  complete  assurance  of  product  quality.  Requirements 
shall  be  included  for  chemical  and  physical  testing  and 
recording  in  connection  with  the  purchase  of  raw 
materials  by  his  suppliers.  The  purchase  orders  must 
also  contain  a  requirement  for  such  suppliers  to  notify 
and  obtain  approval  from  the  contractor  of  changes  in 
design  of  the  products.  Necessary  instructions  should  be 
provided  when  provision  is  made  for  direct  shipment  from 
the  subcontractor  to  Government  activities. 

6 .  MANUFACTURING  CONTROL 

6.L  Materials  and  Materials  Control.  Supplier's  materials, 

process  software,  and  products  shall  be  subjected  to 
inspection  upon  receipt  to  the  extent  necessary  to  assure 
conformance  to  technical  requirements.  Receiving 
inspection  may  be  adjusted  upon  the  basis  of  the  quality 
assurance  program  exercised  by  suppliers.  Evidence  of 
the  suppliers'  satisfactory  control  of  quality  may  be 
used  to  adjust  the  amount  and  kind  of  receiving 
inspection . 

The  quality  program  shall  assure  that  raw  materials  to  be 
used  in  fabrication  or  processing  of  products  conform  to 
the  applicable  physical,  chemical,  and  other  technical 
requirements.  Laboratory  testing  shall  be  employed  as 
necessary.  Suppliers  shall  be  required  by  the 
contractor's  quality  program  to  exercise  equivalent 
control  of  the  raw  materials  and  software  utilized  in  the 
production  of  the  parts  and  items  which  they  supply  to 
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the  eon  l  r.iel  oi  .  Raw  material  awaiting  testing  must  be 
separately  identified  or  segregated  from  already  tested 
and  approved  material  but  ean  be  released  for  initial 
production,  providing  that  identification  and  control  is 
maintained.  Material  tested  and  approved  must  be  kept 
identified  until  such  time  as  its  identity  is  necessarily 
obliterated  by  processing.  Controls  will  be  established 
to  prevent  the  inadvertent  use  of  material  failing  to 
pass  tests. 

Production  Processing  and  Fabrication.  The 
contractor's  quality  program  must  assure  that  all 
machining,  wiring,  batching,  shaping  and  all  basic 
production  operations  of  any  type  together  with  all 
processing  and  fabricating  of  any  type  is  accomplished 
under  controlled  conditions.  Controlled  conditions 
include  documented  work  instructions,  adequate  production 
equipment,  and  any  special  working  environment. 

Documented  work  instructions  are  considered  to  be  the 
criteria  for  much  of  the  production,  processing  and 
fabrication  work.  These  documented  work  instructions  may 
be  hard  copy  or  in  the  form  of  software  such  as  computer 
tape  or  disc.  They  may  be  transmitted  by  hard  copy,  by  a 
computer  terminal  or  directly  to  the  point  of  use.  These 
instructions  are  the  criteria  for  acceptable  or 
unacceptable  "workmanship".  The  quality  program  will 
effectively  monitor  the  issuance  of  and  compliance  with 
all  of  these  work  instructions. 

Physical  examination,  measurement  or  tests  of  the 
material  or  products  processed  must  be  conducted  under 
controlled  conditions.  When  automated  equipment  and 
processes  are  used  conformance  to  requirements  may  be 
determined  without  physical  examination.  When  this 
occurs  the  automated  equipment  and  processes  shall  be 
monitored  sufficiently  to  assure  control.  Both  physical 
inspection  and  process  monitoring  shall  be  provided  when 
control  is  inadequate  without  both, or  when  contract  or 
specification  requires  both. 
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Inspection  and  monitoring  oE  processed  material  or 
products  shall  be  accomplished  in  any  suitable  systematic 
manner  selected  by  the  contractor.  Methods  of  inspection 
and  monitoring  shall  be  corrected  any  time  their 
unsuitability  with  reasonable  evidence  is  demonstrated. 
Adherence  to  selected  methods  for  inspection  and 
monitoring  shall  be  complete  and  continuous.  Corrective 
measures  shall  be  taken  when  noncompl iance  occurs. 

Inspection  by  machine  operators,  automated  gages,  moving 
line  or  lot  sampling,  setup  or  first  piece  approval, 
production  line  inspection  station,  inspection  or  test 
department,  roving  inspectors  --  any  other  type  of 
inspection  —  shall  be  employed  in  any  combination 
desired  by  the  contractor  which  will  adequately  and 
efficiently  protect  product  quality  and  the  integrity  of 
processing. 

Criteria  for  approval  and  rejection  shall  be  provided  for 
all  inspection  of  product  and  monitoring  of  methods, 
equipment,  and  personnel.  Means  for  identifying  rejected 
product  shall  be  provided. 

Cert. i  in  chemical,  me ta ?  I urg i ca t ,  biological,  sonic, 
electronic,  and  r  id iol og ica 1  processes  are  of  so  complex 
and  specialized  a  nature  that  much  more  than  the  ordinary 
detailing  or  work  document -it  ion  is  required.  In  effect, 
such  processing  may  require'  an  entire  work  specification 
as  contracted  with  the  normal  work  operation  instructions 
established  in  normal  plant-wide  standard  production 
control  issuances  such  as  job  operation  routing  books  and 
the  like.  For  these  special  processes,  the  contractors’ 
quality  program  shall  assure  that  the  process  control 
procedures  or  speci f ications  are  adequate  and  that 
processing  environments  and  the  certifying,  inspection, 
authorization  and  monitoring  of  such  processes  to  the 
special  degree  necessary  for  these  ultraprecise  and 
super-complex  work  functions  are  provided. 

6.3  Completed  Item  Inspection  and  Testing.  The  quality 

program  shall  assure  that  there  is  a  system  for  final 
inspection  and  test  of  completed  products.  Such  testing 
shall  provide  a  measure  of  the  overall  quality  of  the 
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completed  product  and  shall  be  performed  so  that  it 
simulates,  to  a  sufficient  degree,  product  end  use  and 
functioning.  Such  simulation  frequently  involves 
appropriate  life  and  endurance  tests  and  qualification 
testing.  Final  inspection  and  testing  shall  provide  for 
reporting  to  designers  any  unusual  difficulties, 
deficiencies  or  questionable  conditions.  When 
modifications,  repairs  or  replacements  are  required  after 
final  inspection  or  testing,  there  shall  be  reinspection 
and  retesting  of  any  characteristics  affected. 

Handling,  Storage  and  Delivery.  The  quality  program 
shall  provide  for  adequate  work  and  inspection 
instructions  for  handling,  storage,  preservation, 
packaging,  and  shipping  to  protect  the  quality  of 
products  and  prevent  damage,  loss  deterioration, 
degradation,  or  substitution  of  products.  With  respect 
to  handling,  the  quality  program  shall  require  and 
monitor  the  use  of  procedures  to  prevent  handling  damage 
to  articles.  Handling  procedures  of  this  type  include 
the  use  of  special  crates,  boxes,  containers, 
transportation  vehicles  and  any  other  facilities  for 
materials  handling.  Means  shall  be  provided  for  any 
necessary  protection  against  deterioration  or  damage  to 
products  in  storage.  Periodic  inspection  for  the 
prevention  and  results  of  such  deterioration  or  damage 
shall  be  provided.  Products  subject  to  deterioration  or 
corrosion  during  fabrication  or  interim  storage  shall  be 
cleaned  and  preserved  by  methods  which  will  protect 
against  such  deterioration  or  corrosion.  When  necessary, 
packaging  designing  and  packaging  shall  include  means  for 
accommodating  and  maintain  critical  environments  within 
packages,  e.g.,  moisture  content  levels,  gas  pressures. 
The  quality  program  shall  assure  that  when  such  packaging 
environments  must  be  maintained,  packages  are  labeled  to 
indicate  this  condition.  The  quality  program  shall 
monitor  shipping  work  to  assure  that  products  shipped  are 
accompanied  with  required  shipping  and  technical 
documents  and  that  compliance  with  Interstate  Commerce 
Commission  rules  and  other  applicable  shipping 
regulations  is  effected  to  assure  safe  arrival  and 
identification  at  destination.  In  compliance  with 
contractual  requirements,  the  quality  program  shall 
include  monitoring  provisions  for  protection  of  the 
quality  of  products  during  transit. 
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Nonconforming  Material.  The  contractor  shall  establish 
and  maintain  an  effective  and  positive  system  for 
controlling  nonconforming  material,  including  procedures 
for  its  identification,  segregation,  and  disposition. 
Repair  of  nonconforming  material  shall  be  in  accordance 
with  documented  procedures  acceptable  to  the  Government. 
The  acceptance  of  nonconforming  supplies  is  a  prerogative 
of  and  shall  be  as  prescribed  by  the  Government  and  may 
involve  a  monetary  adjustment.  All  nonconforming 
supplies  shall  be  positively  identified  to  prevent 
unauthorized  use,  shipment  and  intermingling  with 
conforming  supplies.  Holding  areas  or  procedures 
mutually  agreeable  to  the  contractor  and  the  Government 
Representative  shall  be  provided  by  the  contractor.  The 
contractor  shall  make  known  to  the  Government  upon 
request  the  data  associated  with  the  costs  and  losses  in 
connection  with  scrap  and  with  rework  necessary  to 
reprocess  nonconforming  material  to  make  it  conform 
completely . 

Statistical  Quality  Control  and  Analysis.  In  addition 
to  statistical  methods  required  by  the  contract, 
statistical  planning,  analysis,  test  and  quality  control 
procedures  may  be  utilized  whenever  such  procedures  are 
suitable  to  maintain  the  required  control  of  quality. 
Sampling  plans  may  be  used  when  tests  are  destructive,  or 
when  the  records,  inherent  characteristics  of  the  product 
or  the  noncritical  application  of  the  product,  indicate 
that  a  reduction  in  inspection  or  testing  can  be  achieved 
without  jeopardizing  quality  (critical  and  finite  life 
items  will  normally  be  inspected  100%).  The  contractor 
may  employ  sampling  inspection  in  accordance  with 
applicable  military  standards  and  sampling  plans  (e.g., 
from  MIL-STD-10  5D,  MIL-STD-4  14  ,  or  Handbooks  H  106,  107, 
108).  If  the  contractor  uses  other  sampling  plans,  they 
shall  be  subject  to  review  by  the  cognizant  Government 
Representative.  Any  sampling  plan  used  shall  provide 
valid  confidence  and  quality  levels. 

Indication  of  Inspection  Status.  The  contractor  shall 
maintain  a  positive  system  for  identifying  the 
inspection  status  of  products.  Identification  may  be 
accomplished  by  means  of  stamps,  tags,  routing  cards. 
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move  tickets,  tote  bo:,  cards,  bar  codes  or  other 
acceptable  control  devices .  Such  controls  shall  be  of  a 
design  distinctly  different  from  Government  inspection 
identi f i cat  ion  . 

6.8  Audits.  The  contractor  shall  maintain  a  quality  audit 
plan  to  assure  that  products  meet  contract  quality 
commitments  and  fulfill  the  customer's  quality 
requirements.  The  audits  shall  cover  the  complete 
production  process  for  which  the  contractor  is 
responsible . 

7 .  COORDINATED  GOVERNMENT/CONTRACTOR  ACTIONS 

7.1  Government  Inspection  at  Subcontractor  or  Vendor 
Facilities.  The  Government  reserves  the  right  to 
inspect  at  source  supplies  or  services  not  manufactured 
or  performed  with,  the  contractor's  facility.  Government 
inspection  shall  not  constitute  acceptance;  nor  shall  it 
in  any  way  replace  contractor  inspection  or  otherwise 
relieve  the  contractor  of  his  responsibility  to  furnish 
an  acceptable  end  item.  The  purpose  of  this  inspection 
is  to  assist  the  Government  Representative  at  the 
contractor's  facility  to  determine  the  conformance  of 
supplies  or  services  with  contract  requirements.  Such 
inspection  can  only  be  requested  by  or  under 
authorization  of  the  Government  Representative.  When 
Government  inspection  is  required,  the  contractor  shall 
add  to  his  purchasing  document  the  following  statement; 

"Government  inspection  is  required  prior  to  shipment 
from  your  plant.  Upon  receipt  of  this  order, 
promptly  notify  the  Government  Representative  who 
normally  services  your  plant  so  that  appropriate 
planning  for  Government  inspection  can  be 
accompl i shed . " 

When,  under  authorization  of  the  Government 
Representative,  copies  of  the  purchasing  document  are  to 
be  furnished  directly  by  the  subcontractor  or  vendor  to 
the  Government  Representative  at  his  facility  rather  than 
through  Government  channels,  the  contractor  shall  add  to 
his  purchasing  document  a  statement  substantially  as 
follows : 
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"On  receipt  of  this  order,  promptly  furnish  a  copy  to 
the  Government  Representative  who  normally  services 
your  plant,  or,  if  none,  to  the  nearest  Army,  Navy, 

Air  Force,  or  Defense  Supply  Agency  inspection 
office.  In  the  event  the  representative  or  office 
cannot  be  located,  our  purchasing  agent  should  be 
notified  immediately." 

All  documents  and  referenced  data  for  purchases  applying 
to  a  Government  contract  shall  be  available  for  review  by 
the  Government  Representative  to  determine  compliance 
with  the  requirements  for  the  control  of  such  purchases. 
Copies  of  purchasing  documents  required  for  Government 
purposes  shall  be  furnished  in  accordance  with  the 
instructions  of  the  Government  Representative.  The 
contractor  shall  make  available  to  the  Government 
Representative  reports,  either  as  hard  copy,  soft  copy, 
or  by  access  to  the  computer  data  base,  of  any 
nonconformance  found  on  Government  source  inspected 
supplies  and  shall  (when  requested)  require  the  supplier 
to  coordinate  with  his  Government  Representative  on 
corrective  action. 

7.2  Government  Property. 

7.2.1  Government-furnished  Material.  When  material  is 

furnished  by  the  Government,  the  contractor's  procedures 
shall  include  at  least  the  following: 

(a)  Examination  upon  receipt,  consistent  with 
practicability  to  detect  damage  in  transit; 

(b)  Inspection  for  completeness  and  proper  type; 

(c)  Periodic  inspection  and  precautions  to  assure 
adequate  storage  conditions  and  to  guard  against 
damage  from  handling  and  deterioration  during  storage; 

(d)  Functional  testing,  either  prior  to  or  after 
installation,  or  both,  as  required  by  contract  to 
determine  satisfactory  operation; 

(e)  Identification  anu  protection  from  improper  use  or 
disposition,  and 

(f)  Verification  of  quantity. 
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7.2.2  Damaged  Government-furnished  Material.  The  contractor 
shall  report  to  the  Government  Representative  any 
Government-f urn ished  material  found  damaged, 
malfunctioning,  or  otherwise  unsuitable  for  use.  In  the 
event  of  damage  or  malfunctioning  during  or  after 
installation,  the  contractor  shall  determine  and  record 
probable  cause  and  necessity  for  withholding  material 
from  use. 


7.2.3  Bailed  Property.  The  contractor  shall,  as  required  by 

the  terms  of  the  Bailment  Agreement,  establish  procedures 
for  the  adequate  storage,  maintenance  and  inspection  of 
bailed  Government  property.  Records  of  all  inspections 
and  maintenance  performed  on  bailed  property  shall  be 
maintained.  These  procedures  and  records  shall  be 
subject  to  review  by  the  Government  Representative. 

8 .  NOTES 


(The  following  information  is  provided  solely  for 
guidance  in  using  this  standard.  It  has  no  contractual 
sign i f icance . ) 


8.1  Intended  Use.  This  standard  will  apply  to  complex 

supplies,  components,  equipments  and  systems  for  which 
the  requirements  of  MIL-I-45208  are  inadequate  to  provide 
needed  quality  assurance.  In  such  cases,  total 
conformance  to  contract  requirements  cannot  be  obtained 
effectively  and  economically  solely  by  controlling 
inspection  and  testing.  Therefore,  it  is  essential  to 
control  work  operations  and  manufacturing  processes  as 
well  as  inspections  and  tests.  The  purpose  of  this 
control  is  not  only  to  assure  that  particular  units  of 
hardware  conform  to  contractual  requirements,  but  also  to 
assure  interface  compatibility  among  these  units  of 
hardware/software  when  they  collectively  comprise  major 
equipments,  subsystems  and  systems. 


8.2  Exemptions.  This  standard  will  not  be  applicable  to 
types  of  supplies  for  which  MIL-I-45208  applies.  The 
following  do  not  normally  require  the  application  of  this 
speci f ication . 
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(a)  Personal  services,  and 

(b)  Research  and  development  studies  of  a  theoretical 
nature  which  do  not  require  fabrication  of  articles. 

8.3  Order  Data.  Procurement  documents  should  specify  the 
title,  number  and  date  of  this  standard. 

Preparing  Activity: 

Air  Force  —  Hq  USAF 

Custodians : 

Army  —  Munitions  Command 
Navy  --  Office  of  Naval  Material 
Air  Force  —  Hq  USAF 
.  DSA  —  Hq  DS A 
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3.2  MIL-STD-109  Additions  -  Alternative  Approach 

3.2.1  Objective 

To  establish  improved  flexibility  with  regards  to  the 
interpretation  of  Quality  Assurance  Program  Management 
terminology.  MIL-STD-109,  which  is  the  glossary  for 
MIL-Q-9858A,  is  the  recommended  mechanism  to  provide  this 
flexibility. 

3.2.2  Glossary  Additions/Revisions _ to  MIL-STD-109 

A  review  of  MIL-Q-9858A  reveals  seven  key  words  or 
phrases  that  when  redefined  would  clarify  the  new 
automated  environment.  Those  key  words  are: 

o  Objective  Evidence 

o  Records 

o  Copies 

o  Drawings 

o  Work  Instructions 

o  Identification  of  Inspection  Status 

o  Reports 

There  are  several  ways  to  handle  the  definition  of  these 
key  words.  One  would  be  to  include  a  glossary  as 
paragraph  1.6  in  MIL-Q-9858A.  The  recommended  approach, 
however,  is  to  expand  MIL-STD-109  to  include  the 
following  additions  and  revisions. 

3. 2. 2.1  Objective  Evidence  of  Quality  (Additions  to  Present 

Definitions  are  Underlined) 

Any  statement  of  fact,  either  quantitative  or 
qualitative,  pertaining  to  the  quality  of  a  product 
or  services  based  on  observations,  measurements,  or 
tests  which  can  be  verified.  (Evidence  will  be 
expressed  in  terms  of  specific  quality  requirements 
or  characteristics.  These  characteristics  are 
defined  in  drawings,  specifications,  and  other 
documents  which  describe  the  item,  process,  or 
procedure  and  may  be  presented  on  tapes,  discs  or 
other  electronic  media  capable  o f _be i ng  d i splayed  on 
a  computer  terminal  or  video  screen.)- 
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3. 2.  2. 2  Records  (New) 

Records  are  an  official  account  of  an  activity  kept 
for  future  use.  They  may  on  paper  (hard  copy)  such 
as  written  material,  drawings,  pictures,  sketches, 
charts,  graphs,  and  similar  data  or  they  may  be  on 
film,  tapes,  discs,  and  other  electronic  media  (soft 
copy)  capable  of  being  displayed  on  a  computer 
terminal  or  video  screen. 

3. 2. 2. 3  Copies  (New) 

The  requirement  of  furnishing  the  Government 
Representative  "copies"  of  records  and  other 
objective  evidence  shall  be  satisfied  by  furnishing 
a  paper  or  hardcopy,  or  by  furnishing  a  tape  or  disc 
(soft  copy)  capable  of  being  viewed  on  a  video 
screen.  The  requirement  may  also  be  satisfied  by 
providing  the  Government  Representative  access  to  a 
computer  terminal  or  video  screen  where  the 
information  can  be  seen.  1 

3. 2. 2.4  Drawings  (New) 

Drawings  are  one  part  of  design  documentation.  They 
relate  to  the  design  of  a  product,  machine,  tool,  or 
similai  device.  The  design  may  be  documented 
through  software  stored  on  computer  tape,  disc,  or 
other  suitable  media.  In  many  cases  the  design 
documentation  or  drawing  will  be  transmitted  to  the 
point  of  use  without  being  transcribed  on  paper.  In 
any  case,  drawings  or  design  documentation  will  be 
controlled  in  such  a  way  as  to  assure  effective 
introduction  points  and  the  removal  of  all  obsolete 
drawings  from  points  of  use. 

3. 2. 2.5  Work  Instructions  (New) 

Work  Instructions  are  detailed  descriptions  of  the 
manufacturing  process  or  task.  They  may  be  in  hard 
copy  form  or  in  software  form  such  as  computer  tapes 
or  discs.  Work  Instructions  may  be  transmitted  to 
an  operator  for  use  or  transmitted  directly  to  a 
machine.  Storage  of  Work  Instructions  may  be  either 
as  hardcopy  or  as  software. 
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3.  2.  2.6  ldenti  f  ic.ition  of  Inspection  Status  (New) 

The  contractor  is  required  to  maintain  a  system  for 
identifying  the  inspection  status  of  all  production 
materials  and  products. 

This  identification  may  be  accomplished  by  means  of 
stamps,  tags,  routing  cards,  move  tickets,  tote  box 
cards,  bar  codes,  or  other  acceptable  techniques. 
The  identification  may  also  be  accomplished  through 
automated  routing  or  other  automated  control  of 
material  movement. 

3 . 2 . 2 . 7  Reports  ( New ) 

Reports  may  be  either  in  hardcopy  or  software  form. 
If  the  report  is  in  software  form  the  contractor 
must  make  the  report  available  to  the  Government 
Representative  as  a  tape,  disc,  or  other  suitable 
device,  or  the  Government  Representative  must  be 
able  to  view  the  report  at  a  suitable  computer 
terminal  or  video  screen. 
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3 . 3  MIL-Q-9858A  Rewrite  -  Alternative  Approach 

3.3.1  Ob jec  t ive 

To  establish  an  incremental  improvement  that  will  assure 
product  quality  while  maintaining  the  integrity  of 
MIL-Q-9858A  requirements.  Because  of  the  wide  acceptance 
of  MIL-Q-9858A  throughout  industry  and  the  DoD,  any 
initiatives  for  drastic  change  would  be  an  unacceptable 
approach  for  improvement.  The  following  changes  are 
deemed  appropriate  for  this  approach. 

3.3.2  MIL-Q-9858A  Rewrite 


The  following  section  contains  a  recommended  rewrite  of 
MIL-Q-9858A  for  the  evolutionary  improvement  of  Quality 
Assurance  Program  Management.  Recommended  additions  are 
underlined. 
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RECOMMENDATIONS  FOR  CHANGES  IN  MIL-Q-9858A 
TO  MEET  NEEDS  OF  AUTOMATED  MANUFACTURING  ENVIRONMENTS 
MILITARY  SPECIFICATIONS 
QUALITY  PROGRAM  REQUIREMENTS 


SCOPE 

Applicability.  This  specification  shall  apply  to  all 
supplies  (including  equipments,  subsystems  and  systems) 
or  services  when  referenced  in  the  item  specification, 
contract  or  order. 

Contractual  Intent.  This  specification  requires  the 
establishment  of  a  quality  program  by  the  contractor  to 
assure  compliance  with  the  requirements  of  the  contract. 
The  program  and  procedures  used  to  implement  this 
specification  shall  be  developed  by  the  contractor.  The 
quality  program,  including  procedures,  processes  and 
product  shall  be  documented  in  whatever  manner  is 
appropr  iate  and  shall  be  subject  to  review  "by  the 
government  representative.  The  quality  program  is 
subject  to  the  disapproval  of  the  Government 
Representative  whenever  the  contractor’s  procedures  do 
not  accomplish  their  objectives.  The  Government,  at  its 
option,  may  furnish  written  notice  of  the  acceptability 
of  the  contractor's  quality  program. 

Summary.  An  effective  and  economical  quality  program, 
planned  and  developed  in  consonance  with  the  contractor’s 
other  administrative  and  technical  programs,  is  required 
by  this  specification.  Design  of  the  program  shall  be 
based  upon  consideration  of  the  technical  manufacturing 
aspects  of  production  and  related  engineering  design  and 
materials.  The  program  shall  assure  adequate  quality 
throughout  all  areas  of  contract  performance;  for 
example,  design,  development,  fabrication,  processing, 
assembly,  inspection,  test,  maintenance,  packaging, 
shipping,  storage,  and  site  installation. 
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All  supplies  and  services  under  the  contract,  whether 
manufactured  or  performed  within  the  contractor's  plant 
or  at  any  other  source,  shall  be  controlled  at  all  points 
necessary  to  assure  conformance  to  contractual 
requirements.  The  program  shall  provide  for  the 
prevention  and  ready  detection  of  discrepancies  and  for 
timely  and  positive  corrective  action.  The  contractor 
shall  make  objective  evidence  of  quality  conformance 
readily  available  to  the'  Government  Representative. 
Ctojective  evidence  may  be  in  form  of  written  material. 


■nmn 


Instructions  and  records  for  quality  must  be  controlled. 


The  authority  and  responsibility  of  those  in  charge  of 
the  design,  production,  testing,  and  inspection  of 
quality  shall  be  clearly  stated.  The  program  shall 
facilitate  determinations  of  the  effects  of  quality 
deficiencies  and  quality  costs  on  price.  Facilities  and 
standards  such  as  drawings,  engineering  changes, 
measuring  equipment  and  the  like  which  are  necessary  for 
the  creation  of  the  required  quality  shall  be  effectively 
managed.  The  program  shall  include  an  effective  control 
of  purchased  materials  and  subcontracted  work. 
Manufacturing,  fabrication  and  assembly  work  conducted 
within  the  contractor's  plant  shall  be  controlled 
completely.  The  quality  program  shall  also  include 
effective  execution  of  responsibilities  shared  jointly 
with  the  Government  or  related  to  Government  functions, 
such  as  control  of  Government  property  and  Government 
source  inspection. 


4  Relation  to  Other  Contract  Requirements.  This 

specification  and  any  procedure  or  document  executed  in 
implementation  thereof,  shall  be  in  addition  to  and  not 
in  derogation  of  other  contract  requirements.  The 
quality  program  requirements  set  forth  in  this 
specification  shall  be  satisfied  in  addition  to  all 
detail  requirements  contained  in  the  statement  of  work  or 
in  other  parts  of  the  contract.  The  contractor  is 
responsible  for  compliance  with  all  provisions  of  the 
contract  and  for  furnishing  specified  supplies  and 
services  which  meet  all  the  requirements  of  the 
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contract.  If  any  iricons  is  tency  exists  between  the 
contriict  schedule  or  its  general  provisions  and  this 
specification,  the  contract  schedule  and  the  general 
provisions  shall  control.  The  contractor’s  quality 
program  shall  be  planned  and  used  in  a  manner  to  support 
reliability  effectively. 

1.5  Relation  to  MIL-I-45208.  This  specification  contains 
requirements  in  excess  of  those  in  specification 
MIL-I-45208,  Inspection  System  Requirements,  inasmuch  as 
total  conformance  to  contract  requirements  is  obtained 
best  by  controlling  work  operations,  manufacturing 
processes  as  well  as  inspections  and  tests. 

2.  SUPERSEDING,  SUPPLEMENTATION  AND  ORDERING 

2.1  Applicable  Documents.  The  following  documents  of  the 
issue  in  effect  on  date  of  the  solicitation  form  a  part 
of  this  specification  to  the  extent  specified  herein. 

SPECIFICATIONS 

Military 

MIL-I-45208  —  Inspection  System  Requirements 

MIL-STD-4 5662  —  Calibration  System  Requirements 

2.2  Amendments  and  Revisions.  Whenever  this  specification 
is  amended  or  revised  subsequent  to  its  contractually 
effective  date,  the  contractor  may  follow  or  authorize 
his  subcontractors  to  follow  the  amended  or  revised 
document  provided  no  increase  in  price  or  fee  is 
required.  The  contractor  shall  not  be  required  to  follow 
the  amended  or  revised  document  except  as  a  change  in 
contract.  If  the  contractor  elects  to  follow  the  amended 
or  revised  document,  he  shall  notify  the  Contracting 
Officer,  in  writing,  of  this  election.  When  the 
contractor  elects  to  follow  the  provisions  of  an 
amendment  or  revision,  he  must  follow  them  in  full. 

2.3  Ordering  Government  Documents.  Copies  of 
specification  ,  standards  and  drawings  required  by 
contractors  in  connection  with  specific  procurements  may 
be  obtained  from  the  procuring  agency,  or  as  otherwise 
directed  by  the  Contracting  Officer. 
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QUALITY  PROGRAM  MANAGEMENT 

Organization.  Effective  management  for  quality  shall 
be  clearly  prescribeu  by  the  contractor.  Personnel 
performing  quality  functions  shall  have  sufficient, 
well-defined  responsibility,  authority  and  the 
organizational  freedom  to  identify  and  evaluate  quality 
problems  and  to  initiate,  recommend  or  provide 
solutions.  Management  regularly  shall  review  the  status 
and  adequacy  of  the  quality  program.  The  term  "quality 
program  requirements"  as  used  herein  identifies  the 
collective  requirements  of  this  specification.  It  does 
not  mean  that  the  fulfillment  of  the  requirements  of  this 
speci f ication  is  the  responsibility  of  any  single 
contractor's  organization,  function  or  person. 

Initial  Quality  Planning.  The  contractor,  during  the 
earliest  practical  phase  of  contract  performance,  shall 
conduct  a  complete  review  of  the  requirements  of  the 
contract  to  identify  and  make  timely  provision  for  the 
special  controls,  processes,  test  equipments,  fixtures, 
tooling  and  skills  required  for  assuring  product 
quality.  This  initial  planning  will  organize  the  need 
and  provide  for  research,  when  necessary,  to  update 
inspection  and  testing  techniques,  instrumentation  and 
correlation  of  inspection  and  test  results  with 
manufacturing  methods  and  processes.  This  planning  will 
also  provide  appropriate  review  and  action  to  assure 
compatibility  of  manufacturing,  inspection,  testing  and 
documentation  . 

Work  Instructions.  The  quality  program  shall  assure 
that  all  work  affecting  quality  (including  such  things  as 
purchasing,  handling,  machining,  assembling,  fabricating, 
processing,  inspection,  testing,  modification, 
installation,  and  any  other  treatment  of  product, 
facilities,  standards  or  equipment  from  the  ordering  of 
materials  to  dispatch  of  shipments)  shall  be  prescribed 
in  clear  and  complete  documented  instructions  of  a  type 
appropriate  to  the  circumstances.  Work  instructions  may 
be  in  hard  copy  form  such  as  written  instructions  on 
paper  or  film,  or  in  soft  copy  form  such  as  computer 


tapes  and  discs.  Work  instructions  may  be  transmitted  to 
an  operator  or  directly  to  a  machine.  Such  instructions 
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shall  provide  the  criteria  for  performing  the  work 
functions  and  they  shall  be  compatible  with  acceptance 
criteria  for  workmanship.  The  instructions  are  intended 
also  to  serve  for  supervising,  inspecting  and  managing 
work.  The  preparation  and  maintenance  of  and  compliance 
with  work  instructions  shall  be  monitored  as  a  function 
of  the  quality  program. 

Records.  The  contractor's  quality  assurance  program 
documentation  and  planning  shall  focus  on  preventing 
quality  problems.  These  records,  either  as  hard  copy  or 
as  soft  copy  such  as  computer  tapes  or  discs,  shall  be 
available  for  review  by  the  Government  Representative  and 
copies  of  individual  records  shall  be  furnished  him  upon 
request.  The  requirement  of  furnishing  the  Government 
Representative  copies  of  records  shall  be  satisfied  by 
furnishing  a  hard  copy  of  the  records  or  by  furnishing  a 
soft  copy,  such  as  a  tape  or  disc,  capable  of  being 
viewed  at  a  computer  terminal  or  a  video  screen.  The 
requirement  may  also  be  satisfied  by  providing  the 
Government  Representative  access  to  a  computer  terminal 
or  a  video  screen  where  the  information  can  be  seen. 
Records  are  considered  one  of  the  principal  forms  of 
objective  evidence  of  quality.  The  quality  program  shall 
assure  that  records  are  complete  and  reliable. 

Inspection  and  testing  records  shall,  as  a  minimum, 
indicate  the  nature  of  the  observations  together  with  the 
number  of  observations  made  and  the  number  and  type  of 
deficiencies  found.  Also,  records  for  monitoring  work 
performance  and  for  inspection  and  testing  shall  indicate 
the  acceptability  of  work  or  products  and  the  action 
taken  in  connection  with  deficiencies.  The  quality 
program  shall  provide  for  the  analysis  and  use  of  records 
as  a  basis  for  management  action. 

Corrective  Action.  The  quality  program  shall  detect 
promptly  and  correct  assignable  conditions  adverse  to 
quality.  Design,  purchasing,  manufacturing,  testing  or 
other  operations  which  could  result  in  or  have  resulted 
in  defective  supplies,  services,  facilities,  technical 
data,  standards  or  other  elements  of  contract  performance 
which  could  create  excessive  losses  or  costs  must  be 
identified  and  changed  as  a  result  of  the  quality 
program.  Corrective  action  will  extend  to  the 
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performance  of  all  suppliers  and  vendors  and  will  be 
responsive  to  data  and  product  forwarded  from  users. 
Corrective  action  shall  include  as  a  minimum: 

(a)  Analysis  of  data  and  examination  of  product  scrapped 
or  reworked  to  determine  extent  and  causes; 

(b)  Analysis  of  trends  in  processes  or  performance  of 
work  to  prevent  nonconforming  product;  and 

(c)  Introduction  of  required  improvements  and 
corrections,  an  initial  review  of  the  adequacy  of 
such  measures  and  monitoring  of  the  effectiveness  of 
corrective  action  taken. 

Costs  Related  to  Quality.  The  contractor  shall 
maintain  and  use  quality  cost  data  as  a  management 
element  of  the  quality  program.  These  data  shall  serve 
the  purpose  of  identifying  the  cost  of  both  the 
prevention  and  correction  of  nonconforming  supplies 
(e.g.,  labor  and  material  involved  in  material  spoilage 
caused  by  defective  work,  correction  of  defective  work 
and  for  quality  control  exercised  by  the  contractor  at 
subcontractor's  or  vendor's  facilities).  The  specific 
quality  cost  data  to  be  maintained  and  used  will  be 
determined  by  the  contractor.  These  data  shall,  on 
request,  be  identified  and  made  available  for  review  by 
the  Government  Representative. 

FACILITIES  AND  STANDARDS 

Drawings,  Documentation  and  Changes.  A  procedure  shall 
be  maintained  that  concerns  itself  with  the  adequacy,  the 
completeness  and  the  currentness  of  design  documents  and 
with  the  control  of  changes  in  design.  Design  documents 
include  drawings  and  specifications  either  in  hard  copy 
form  or  in  software  form  such  as  computer  tapes  or 
discs.  With  respect  to  the  currentness  of  des iqn 
documents  and  changes,  the  contractor  shall  assure  that 
requirements  for  the  effectivity  point  of  changes  are  met 
and  that  obsolete  design  documents  and  change 
requirements  are  removed  from  all  points  of  issue  and 
use.  Some  means  of  recording  the  effective  points  shall 
be  employed  and  be  available  to  the  Government. 
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With  respect  to  design  documents .  a  procedure  shall  be 
maintained  that  shall  provide  for  the  evaluation  of  their 
engineering  adequacy  of  proposed  changes.  The  evaluation 
shall  encompass  both  the  adequacy  in  relation  to  standard 
engineering  and  design  practices  and  the  adequacy  with 
respect  to  the  design  and  purpose  of  the  product  to  which 
the  drawing  relates. 

With  respect  to  supplemental  specifications,  process 
instructions,  production  engineering  instructions  and 
work  instructions  relating  to  a  particular  design,  the 
contractor  shall  be  responsible  for  a  review  of  their 
adequacy,  currentness  and  completeness.  The  quality 
program  must  provide  complete  coverage  of  all  information 
necessary  to  produce  an  article  in  complete  conformity 
with  requirements  of  the  design. 


The  quality  program  shall  assure  that  there  is  complete 
compliance  with  contract  requirements  for  proposing, 
approving,  and  effecting  of  engineering  changes.  The 
quality  program  shall  provide  for  monitoring  effectively 
compliance  with  contractual  engineering  changes  requiring 
approval  by  Government  design  authority.  The  quality 
program  shall  provide  for  monitoring  effectively  the 
design  documentation  changes  of  lesser  importance  not 
requiring  approval  by  Government  design  authorities. 


Delivery  of  correct  design  documentation  and  change 
information  to  the  Government  in  connection  with  data 
acquisition  shall  be  an  integral  part  of  the  quality 
program.  This  includes  full  compliance  with  contract 
requirements  concerning  rights  and  data  both  proprietary 
and  other.  The  quality  program's  responsibility  for 
drawings  and  changes  extend  to  the  drawings  and  changes 
provided  by  the  subcontractors  and  vendors  for  the 
con  tract . 


4.2  Measuring  and  Testing  Equipment.  The  contractor  shall 
provide  and  maintain  gages  and  other  measuring  and 
testing  devices  necessary  to  assure  that  supplies  conform 
to  technical  requirements.  These  devices  shall  be 
calibrated  against  certified  measurement  standards  which 
have  known  valid  relationships  to  national  standards  at 
established  periods  to  assure  continued  accuracy.  The 
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objective  is  to  assure  that  inspection  and  test  equipment 
is  adjusted,  replaced  or  repaired  before  it  becomes 
inaccurate.  The  calibration  of  measuring  and  testing 
equipment  shall  be  in  conformity  with  military 
specification  MIL-STD-4 5662 .  In  addition,  the  contractor 
shall  insure  the  use  of  only  such  subcontractor  and 
vendor  sources  that  depend  upon  calibration  systems  which 
effectively  control  the  accuracy  of  measuring  and  testing 
equipment . 

4.3  Production  Tooling  Used  as  Media  of  Inspection.  When 
production  jigs,  fixtures,  tooling  masters,  templates, 
patterns  and  such  other  devices  are  used  as  media  of 
inspection,  they  shall  be  proved  for  accuracy  prior  to 
release  for  use.  These  devices  shall  be  proved  again  for 
accuracy  at  intervals  formally  established  in  a  manner  to 
cause  their  timely  adjustment,  replacement  or  repair 
prior  to  becoming  inaccurate. 

4.4  Use  of  Contractor's  Inspection  Equipment.  The 
contractor's  gages,  measuring  and  testing  devices  shall 
be  made  available  for  use  by  the  Government  when  required 
to  determine  conformance  with  contract  requirements.  If 
conditions  warrant,  contractor’s  personnel  shall  be  made 
available  for  operation  of  such  devices  and  for 
verification  of  their  accuracy  and  condition. 

4.5  Advanced  Metrology  Requirements.  The  quality  program 
shall  include  timely  identification  and  report  to  the 
Contracting  Officer  of  any  precision  measurement  need 
exceeding  the  known  state  of  the  art. 

5.  CONTROL  OF  PURCHASES 

5.1  Responsibility.  The  contractor  is  responsible  for 

assuring  that  all  supplies  and  services  procured  from  his 
suppliers  (subcontractors  and  vendors)  conform  to  the 
contract  requirements.  The  selection  of  sources  and  the 
nature  and  extent  of  control  exercised  by  the  contractor 
shall  be  dependent  upon  the  type  of  supplies,  his 
supplier's  demonstrated  capability  to  perform,  and  the 
quality  evidence  made  available.  To  assure  an  adequate 
and  economical  control  of  such  material,  the  contractor 
shall  utilize  to  the  fullest  extent  objectives  evidence 
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of  quality  furnished  by  his  suppliers.  When  the 
Government  elects  to  perform  inspection  at  a  supplier’s 
plant,  such  inspection  shall  not  be  used  by  contractors 
as  evidence  of  effective  control  of  quality  by  such 
suppliers.  The  inclusion  of  a  product  on  the  Qualified 
Products  List  only  signifies  that  at  one  time  the 
manufacturer  made  a  product  which  met  specification 
requirements.  It  does  not  relieve  the  contractor  of  his 
responsibility  for  furnishing  supplies  that  meet  all 
specification  requirements  or  for  the  performance  of 
specified  inspections  and  tests  for  such  material.  The 
effectiveness  and  integrity  of  the  control  of  quality  by 
his  suppliers  shall  be  assessed  and  reviewed  by  the 
contractor  at  intervals  consistent  with  the  complexity 
and  quantity  of  product.  Inspection  of  products  upon 
delivery  to  the  contractor  shall  be  used  for  assessment 
and  review  to  the  extent  necessary  for  adequate  assurance 
of  quality.  Test  reports,  inspection  records, 
certificates  and  other  suitable  evidence  relating  to  the 
supplier's  control  of  quality  should  be  used  in  the 
contractor's  assessment  and  review.  The  contractor's 
responsibility  for  the  control  of  purchases  includes  the 
establishment  of  a  procedure  for  (1)  the  selection  of 
qualified  suppliers,  (2)  the  transmission  of  applicable 
design  and  quality  requirements  in  the  Government 
contracts  and  associated  technical  requirements,  (3)  the 
evaluation  of  the  adequacy  of  procured  items,  and  (4) 
effective  provisions  for  early  information  feedback  and 
correction  of  nonconformances. 

Purchasing  Data.  The  contractor’s  quality  program 
shall  not  be  acceptable  to  the  Government  unless  the 
contractor  requires  of  his  subcontractors  a  quality 
effort  achieving  control  of  the  quality  of  the  services 
and  supplies  which  they  provide.  The  contractor  shall 
assure  that  all  applicable  requirements  are  properly 
included  or  referenced  in  all  purchase  orders  for 
products  ultimately  to  apply  on  a  Government  contract. 

The  purchase  order  shall  contain  a  complete  description 
of  the  supplies  ordered  including,  by  statement  or 
reference,  all  applicable  requirements  for  manufactur ing , 
inspections,  testing,  packaging,  and  any  requirements  for 
Government  or  contractor  inspections,  qualification  or 
approvals.  Technical  requirements  of  the  following 
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nature  must  be  included  by  statement  or  reference  as  a 
part  of  the  required  clear  description:  all  pertinent 
design  documentation,  engineering  des i gn  change  orders, 
specifications  (including  inspection  system  or  quality 
program  requirements),  reliability,  safety,  weight,  or 
other  special  requirements,  unusually  test  or  inspection 
procedures  or  equipment  and  any  special  revision  or  model 
identification.  The  description  of  products  ordered 
shall  include  a  requirement  for  contractor  inspection  at 
the  subcontractor  or  vendor  source  when  such  action  is 
necessary  to  assure  that  the  contractor's  quality  program 
effectively  implements  the  contractor's  responsibility 
for  complete  assurance  of  product  quality.  Requirements 
shall  be  included  for  chemical  and  physical  testing  and 
recording  in  connection  with  the  purchase  of  raw 
materials  by  his  suppliers.  The  purchase  orders  must 
also  contain  a  requirement  for  such  suppliers  to  notify 
and  obtain  approval  from  the  contractor  of  changes  in 
design  of  the  products.  Necessary  instructions  should  be 
provided  when  provision  is  made  for  direct  shipment  from 
the  subcontractor  to  Government  activities. 

6  .  MANUFACTURING  CONTROL, 

6.1  Materials  and  Materials  Control.  Supplier's  materials 

and  products  shall  be  subjected  to  inspection  upon 
receipt  to  the  extent  necessary  to  assure  conformance  to 
technical  requirements.  Receiving  inspection  may  be 
adjusted  upon  the  basis  of  the  quality  assurance  program 
exercised  by  suppliers.  Evidence  of  the  suppliers' 
satisfactory  control  of  quality  may  be  used  to  adjust  the 
amount  and  kind  of  receiving  inspection. 

The  quality  program  shall  assure  that  raw  materials  to  be 
used  in  fabrication  or  processing  of  products  conform  to 
the  applicable  physical,  chemical,  and  other  technical 
requirements.  Laboratory  testing  shall  be  employed  as 
necessary.  Suppliers  shall  be  required  by  the 
contractor's  quality  program  to  exercise  equivalent 
control  of  the  raw  materials  utilized  in  the  production 
of  the  parts  and  items  which  they  supply  to  the 
contractor.  Raw  material  awaiting  testing  must  be 
separately  identified  or  segregated  from  already  tested 
and  approved  material  but  can  be  released  for  initial 
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production,  providing  that  iden t i f ica t ion  and  control  is 
maintained.  Material  tested  and  approved  must  be  kept 
identified  until  such  time  as  its  identity  is  necessarily 
obliterated  by  processing.  Controls  will  be  established 
to  prevent  the  inadvertent  use  of  material  failing  to 
pass  tests. 

6.2  Production  Processing  and  Fabrication.  The 

contractor's  quality  program  must  assure  that  all 
machining,  wiring,  batching,  shaping  and  all  basic 
production  operations  of  any  type  together  with  all 
processing  and  fabricating  of  any  type  is  accomplished 
under  controlled  conditions.  Controlled  conditions 
include  documented  work  instructions,  adequate  production 
equipment,  and  any  special  working  environment. 

Documented  work  instructions  are  considered  to  be  the 
criteria  for  much  of  the  production,  processing  and 
fabrication  work.  These  documented  work  instructions  may 
be  hard  copy  or  in  the  form  of  software  such  as  computer 
tape  or  disc.  They  may  be  transmitted  by  hard  copy,  by  a 
computer  terminal  or  directly  to  the  point  of  use.  These 
instructions  are  the  criteria  for  acceptable  or 
unacceptable  "workmanship".  The  quality  program  will 
effectively  monitor  the  issuance  of  and  compliance  with 
all  of  these  work  instructions. 

Physical  examination,  measurement  or  tests  of  the 
material  or  products  processed  is  necessary  for  each  work 
operation  and  must  also  be  conducted  under  controlled 
conditions.  If  physical  inspection  of  processed  material 
is  impossible  or  disadvantageous,  indirect  control  by 
monitoring  processing  methods,  equipment  and  personnel 
shall  be  provided.  Both  physical  inspection  and  process 
monitoring  shall  be  provided  when  control  is  inadequate 
without  both, or  when  contract  or  specification  requires 
bo  th . 

Inspection  and  monitoring  of  processed  material  or 
products  shall  be  accomplished  in  any  suitable  systematic 
manner  selected  by  the  contractor.  Methods  of  inspection 
and  monitoring  shall  be  corrected  any  time  their 
unsuitability  with  reasonable  evidence  is  demonstrated. 
Adherence  to  selected  methods  for  inspection  and 
monitoring  shall  be  complete  and  continuous.  Corrective 
measures  shall  be  taken  when  noncompliance  occurs. 
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6.3 


Inspection  by  machine  operators,  automated  gages,  moving 
line  or  lot  sampling,  setup  or  first  piece  approval, 
production  line  inspection  station,  inspection  or  test 
department,  roving  inspectors  --  any  other  type  of 
inspection  --  shall  be  employed  in  any  combination 
desired  by  the  contractor  which  will  adequately  and 
efficiently  protect  product  quality  and  the  integrity  of 
process ing . 

Criteria  for  approval  and  rejection  shall  be  provided  for 
all  inspection  of  product  and  monitoring  of  methods, 
equipment,  and  personnel.  Means  for  identifying  approved 
and  rejected  product  shall  be  provided. 


Certain  chemical,  metallurgical,  biological,  sonic, 
electronic,  and  radiological  processes  are  of  so  complex 
and  specialized  a  nature  that  much  more  than  the  ordinary 
detailing  or  work  documentation  is  required.  In  effect, 
such  processing  may  require  an  entire  work  specification 
as  contracted  with  the  normal  work  operation  instructions 
established  in  normal  plant-wide  standard  production 
control  issuances  such  as  job  operation  routing  books  and 
the  like.  For  these  special  processes,  the  contractors' 
quality  program  shall  assure  that  the  process  control 
procedures  or  specifications  are  adequate  and  that 
processing  environments  and  the  certifying,  inspection, 
authorization  and  monitoring  of  such  processes  to  the 
special  degree  necessary  for  these  ultraprecise  and 
super-complex  work  functions  are  provided. 

Completed  Item  Inspection  and  Testing.  The  quality 
program  shall  assure  that  there  is  a  system  for  final 
inspection  and  test  of  completed  products.  Such  testing 
shall  provide  a  measure  of  the  overall  quality  of  the 
completed  product  and  shall  be  performed  so  that  it 
simulates,  to  a  sufficient  degree,  product  end  use  and 
functioning.  Such  simulation  frequently  involves 
appropriate  life  and  endurance  tests  and  qualification 
testing.  Final  inspection  and  testing  shall  provide  for 
reporting  to  designers  any  unusual  difficulties, 
deficiencies  or  questionable  conditions.  When 
modifications,  repairs  or  replacements  are  required  after 
final  inspection  or  testing,  there  shall  be  reinspection 
and  retesting  of  any  characteristics  affected. 
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6.4  Handling,  Storage  and  Delivery.  The  quality  program 
shall  provide  for  adequate  work  and  inspection 
instructions  for  handling,  storage,  preservation, 
packaging,  and  shipping  to  protect  the  quality  of 
products  and  prevent  damage,  loss  deterioration, 
degradation,  or  substitution  of  products.  With  respect 
to  handling,  the  quality  program  shall  require  and 
monitor  the  use  of  procedures  to  prevent  handling  damage 
to  articles.  Handling  procedures  of  this  type  include 
the  use  of  special  crates,  boxes,  containers, 
transportation  vehicles  and  any  other  facilities  for 
materials  handling.  Means  shall  be  provided  for  any 
necessary  protection  against  deterioration  or  damage  to 
products  in  storage.  Periodic  inspection  for  the 
prevention  and  results  of  such  deterioration  or  damage 
shall  be  provided.  Products  subject  to  deterioration  or 
corrosion  during  fabrication  or  interim  storage  shall  be 
cleaned  and  preserved  by  methods  which  will  protect 
against  such  deterioration  or  corrosion.  When  necessary, 
packaging  designing  and  packaging  shall  include  means  for 
accommodating  and  maintain  critical  environments  within 
packages,  e.g.,  moisture  content  levels,  gas  pressures. 
The  quality  program  shall  assure  that  when  such  packaging 
environments  must  be  maintained,  packages  are  labeled  to 
indicate  this  condition.  The  quality  program  shall 
monitor  shipping  work  to  assure  that  products  shipped  are 
accompanied  with  required  shipping  and  technical 
documents  and  that  compliance  with  Interstate  Commerce 
Commission  rules  and  other  applicable  shipping 
regulations  is  effected  to  assure  safe  arrival  and 
identification  at  destination.  In  compliance  with 
contractual  requirements,  the  quality  program  shall 
include  monitoring  provisions  for  protection  of  the 
quality  of  products  during  transit. 

6.5  Nonconforming  Material.  The  contractor  shall  establish 
and  maintain  an  effective  and  positive  system  for 
controlling  nonconforming  material,  including  procedures 
for  its  identification,  segregation,  and  disposition. 
Repair  or  rework  of  nonconforming  material  shall  be  in 
accordance  with  documented  procedures  acceptable  to  the 
Government.  The  acceptance  of  nonconforming  supplies  is 
a  prerogative  of  and  shall  be  as  prescribed  by  the 
Government  and  may  involve  a  monetary  adjustment.  All 
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nonconforming  supplies  shall  be  positively  identified  to 
prevent  unauthorized  use,  shipment  and  intermingling  with 
conforming  supplies.  Holding  areas  or  procedures 
mutually  agreeable  to  the  contractor  and  the  Government 
Representative  shall  be  provided  by  the  contractor.  The 
contractor  shall  make  known  to  the  Government  upon 
request  the  data  associated  with  the  costs  and  losses  in 
connection  with  scrap  and  with  rework  necessary  to 
reprocess  nonconforming  material  to  make  it  conform 
completely . 

6.6  Statistical  Quality  Control  and  Analysis.  In  addition 
to  statistical  methods  required  by  the  contract, 
statistical  planning,  analysis,  test  and  quality  control 
procedures  may  be  utilized  whenever  such  procedures  are 
suitable  to  maintain  the  required  control  of  quality. 
Sampling  plans  may  be  used  when  tests  are  destructive,  or 
when  the  records,  inherent  characteristics  of  the  product 
or  the  noncritical  application  of  the  product,  indicate 
that  a  reduction  in  inspection  or  testing  can  be  achieved 
without  jeopardizing  quality.  The  contractor  may  employ 
sampling  inspection  in  accordance  with  applicable 
military  standards  and  sampling  plans  (e.g.,  form 
MIL-STD-10 5 ,  MIL-STD-4 14  ,  or  Handbooks  H  106,  107,  108). 
If  the  contractor  uses  other  sampling  plans,  they  shall 
be  subject  to  review  by  the  cognizant  Government 
Representative.  Any  sampling  plan  used  shall  provide 
valid  confidence  and  quality  levels. 

6.7  Indication  of  Inspection  Status.  The  contractor  shall 
maintain  a  positive  system  for  identifying  the  inspection 
status  of  products.  Identification  may  be  accomplished 
by  means  of  stamps,  tags,  routing  cards,  move  tickets, 
tote  box  cards,  bar  codes  or  other  acceptable  control 
devices.  Such  controls  shall  be  of  a  design  distinctly 
different  from  Government  inspection  identi f ication . 

7.  COORDINATED  GOVERNMENT/CONTRACTOR  ACTIONS 

7.1  Government  Inspection  at  Subcontractor  or  Vendor 

Facilities.  The  Government  reserves  the  right  to 
inspect  at  source  supplies  or  services  not  manufactured 
or  performed  with  the  contractor's  facility.  Govenment 
inspection  shall  not  constitute  acceptance;  nor  shall  it 
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in  any  way  replace  contractor  inspection  or  otherwise 
relieve  the  contractor  of  his  responsibility  to  furnish 
an  acceptable  end  item.  The  purpose  of  this  inspection 
is  to  assist  the  Government  Representative  at  the 
contractor's  facility  to  determine  the  conformance  of 
supplies  or  services  with  contract  requirements.  Such 
inspection  can  only  be  requested  by  or  under 
authorization  of  the  Government  Representative.  When 
Government  inspection  is  required,  the  contractor  shall 
add  to  his  purchasing  document  the  following  statement: 

"Government  inspection  is  required  prior  to  shipment 
from  your  plant.  Upon  receipt  of  this  order, 
promptly  notify  the  Government  Representative  who 
normally  services  your  plant  so  that  appropriate 
planning  for  Government  inspection  can  be 
accomplished . . " 

When,  under  authorization  of  the  Government 
Representative,  copies  of  the  purchasing  document  are  to 
be  furnished  directly  by  the  subcontractor  or  vendor  to 
the  Government  Representative  at  his  facility  rather  than 
through  Government  channels,  the  contractor  shall  add  to 
his  purchasing  document  a  statement  substantially  as 
follows: 

"On  receipt  of  this  order,  promptly  furnish  a  copy 
to  the  Government  Representative  who  normally 
services  your  plant,  or,  if  none,  to  the  nearest 
Army,  Navy,  Air  Force,  or  Defense  Supply  Agency 
inspection  office.  In  the  event  the  representative 
or  office  cannot  be  located,  our  purchasing  agent 
should  be  notified  immediately." 

All  documents  and  referenced  data  for  purchases  applying 
to  a  Government  contract  shall  be  available  for  review  by 
the  Government  Representative  to  determine  compliance 
with  the  requirements  for  the  control  of  such  purchases. 
Copies  of  purchasing  documents  required  for  Government 
purposes  shall  be  furnished  in  accordance  with  the 
instructions  of  the  Government  Representative.  The 
contractor  shall  make  available  to  the  Government 
Representative  reports,  either  as  hard  copy,  software,  or 
by  access  to  the  computer  data  base,  of  any 
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nonconformance  found  on  Government-  source  inspected 
supplies  and  shall  (when  requested)  require  the  supplier 
to  coordinate  with  his  Government  Representative  on 
corrective  action. 

7.2  Government  Property. 


7.2.1  Government-furnished  Material.  When  material  is 

furnished  by  the  Government,  the  contractor's  procedures 
shall  include  at  least  the  following: 

(a)  Examination  upon  receipt,  consistent  with 
practicability  to  detect  damage  in  transit; 


(b)  Inspection  for  completeness  and  proper  type; 


!  (c)  Periodic  inspection  and  precautions  to  assure 

adequate  storage  conditions  and  to  guard  against 
damage  from  handling  and  deterioration  during  storage; 


(d) 


1 

I 


Functional  testing,  either  prior  to  or  after 
installation,  or  both,  as  required  by  contract  to 
determine  satisfactory  operation; 


(e)  Identification  and  protection  from  improper  use  or 
disposition,  and 


(f)  Verification  of  quantity. 

7.2.2  Damaged  Government- f urn i shed  Material.  The  contractor 
shall  report  to  the  Government  Representative  any 
Government-furnished  material  found  damaged, 
mal f unct ion ing ,  or  otherwise  unsuitable  for  use.  In  the 
event  of  damage  or  malfunctioning  during  or  after 
installation,  the  contractor  shall  determine  and  record 
probable  cause  and  necessity  for  withholding  material 
from  use. 


7.2.3  Bailed  Property.  The  contractor  shall,  as  required  by 

the  terms  of  the  Bailment  Agreement,  establish  procedures 
for  the  adequate  storage,  maintenance  and  inspection  of 
bailed  Government  property.  Records  of  all  inspections 
and  maintenance  performed  on  bailed  property  shall  be 
maintained.  These  procedures  and  records  shall  be 
subject  to  review  by  the  Government  Representative. 


I 


3-11 


I 


IS  PI  10  51  3000 
10  AUGUST  1903 


8.  NOTES 

(The  following  information  is  provided  solely  for 
guidance  in  using  this  specification.  It  has  no 
contractual  s ign i f icance  .  ) 

8.1  Intended  Use.  This  speci f ication  will  apply  to  complex 
supplies,  components,  equipments  and  systems  for  which 
the  requirements  of  MIL-I-4  5208  are  inadequate  to  provide 
needed  quality  assurance.  In  such  cases,  total 
conformance  to  contract  requirements  cannot  be  obtained 
effectively  and  economically  solely  by  controlling 
inspection  and  testing.  Therefore,  it  is  essential  to 
control  work  operations  and  manufacturing  processes  as 
well  as  inspections  and  tests.  The  purpose  of  this 
control  is  not  only  to  assure  that  particular  units  of 
hardware  conform  to  contractual  requirements,  but  also  to 
assure  interface  compatibility  among  these  units  of 
hardware  when  they  collectively  comprise  major 
equipments,  subsystems  and  systems. 

8.2  Exemptions.  This  specification  will  not  be  applicable 
to  types  of  supplies  for  which  MIL-I-45208  applies.  The 
following  do  not  normally  require  the  application  of  this 
speci f ication . 

(a)  Personal  services,  and 

(b)  Research  and  development  studies  of  a  theoretical 
nature  which  do  not  require  fabrication  of  articles. 

8.3  Order  Data.  Procurement  documents  should  specify  the 
title,  number  and  date  of  this  specification. 


Preparing  Activity: 

Air  Force  —  Hq  USAF 
Custodians : 

Army  —  Munitions  Command 
Navy  —  Office  of  Naval  Material 
Air  Force  —  Hq  USAF 
DSA  —  Hq  DS  A 
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APPENDIX  A 

QUALITY  ASSURANCE  PROGRAM  MANAGEMENT  STANDARD 


DATA  COLLECTION 


Appendix  A  contains  the  presentation  given  at  fact-finding 
visits  and  was  used  to  explain  the  QA/QC  project  before  asking 
for  a  response  to  the  questionnaire  or  having  a  general 
discussion  on  what  should  be  included  in  a  new  quality  assurance 
standard  for  the  automated  environment  of  the  Factory  of  The 
Future. 
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APPENDIX  13 

GOVERNMENT/ INDUSTRY  SURVEY 


This  appendix  contains  the  results  of  the  Government/Industry 
Survey.  The  first  part  (B.l  through  B.3)  summarizes  both  the 
approach  and  the  responses  to  a  questionnaire  that  was  used  to 
gather  information.  Section  B.4  presents  the  conclusions 
derived  from  this  effort. 

Following  this.  Section  B.5  contains  all  of  the  responses  to  the 
questionnaire.  As  decribed  in  Section  B.2.1,  the  questionnaire 
is  broken  into  two  sections.  The  first  section  contains  the 
Quality  Assurance  Ratings.  In  this  section,  the  respondent  is 
asked  to  provide  a  rating  from  "high"  to  "low"  to  a  set  of  four 
questions  arrayed  against  the  six  nodes  of  the  QA/A-0  Assure 
Product  Quality  model.  The  second  section  is  the  Quality 
Assurance  Questionnaire.  This  section  contains  seven  broad 
questions  covering  present  experience  and  forecasted  needs  in 
the  automated  environment  of  the  Factory-of-the-Future . 

Finally,  Section  B.6  contains  the  minutes  from  each  information 
gathering  meeting.  The  sources  for  this  information  are  given 
in  Section  B.2.2. 
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Introduction 

After  completing  the  major  work  on  Task  4.4.2  - 
Understand  the  Problem  (the  initial  phase  in  the  ICAM 
Life  Cycle  which  defines  the  system  environment,  needs 
and  requirements),  our  attention  was  directed  to  the  last 
phase  of  the  program.  Task  4.4.4  -  Establish  Quality 
Assurance  Program  Management  Standard.  As  part  of  Task 
4.4.2,  a  Needs  Analysis  Document  and  a  System 
Requirements  Document  were  developed.  These  documents 
provided  a  great  deal  of  insight  for  Task  4.4.4  but  from 
a  Limited  point  of  view.  Our  coalition  had  focused  on 
Quality  Assurance  in  airframe  manufacturing  and  that 
focus  was  from  the  point  of  view  of  a  Quality 
Program/Project  Manager.  We  felt  it  was  necessary  to 
expand  our  focus  in  two  ways. 

First,  we  wanted  to  gather  data  from  other  Air  Force 
contractors  such  as  those  supplying  electronics.  In 
addition,  we  wanted  data  from  other  DoD  contractors  who 
provided  Army  and  Navy  supplies.  And  of  course,  we 
wanted  input  from  the  Air  Force  and  the  Department  of 
Defense. 

Second,  we  wanted  to  hear  from  the  very  highest  ranking 
Quality  Assurance  officers  in  industry  and  in  the 
military.  We  wanted  to  combine  data  from  the  "policy 
makers"  with  the  working  level  data  already  accumulated. 

In  addition  to  simply  expanding  our  data  base,  we  wanted 
to  get  responses  to  questions  directed  specifically  at  a 
new  Quality  Assurance  Program  Management  Standard. 

Approach 

Our  approach  to  meeting  these  objectives  was  to  develop  a 
questionnaire,  shown  in  Figures  B-l  through  B-3,  and  to 
establish  a  series  of  data  gathering  meetings. 
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U.2.1  Uiji's I  ionnairo 

The  questionnaire  is  broken  into  two  sections.  The  first 
section  contains  the  Quality  Assurance  Ratings.  In  this 
section  the  respondent  is  asked  to  provide  a  rating  from 
"high"  to  “low"  to  a  set  of  four  questions  arrayed 
against  the  six  nodes  of  the  QA/A-0  Assure  Product 
Quality  model.  The  second  section  is  the  Quality 
Assurance  Questionnaire.  This  section  contains  seven 
broad  questions  covering  present  experience  and 
forecasted  needs  in  the  automated  environment  of  the 
Factory-of-the-Futuro. 

B.2.2  Contacts 


Data  and  information  was  collected  from  the  following 
sources : 

Industry ; 

Vought  Corporation 
General  Dynamics 
Grumman 
Hughes 

McDonnell  Douglas 
Bendix 

Texas  Instruments 
Boeing 
Honeywell* 

Lockheed  Georgia* 

Martin  Marietta* 

TRW* 

United  Technologies* 

General  Electric; 

Aerospace  Electronic  Systems,  Utica 
Aircraft  Engine,  Evendale 

Simulation  and  Control  Systems,  Daytona  Beach 
Ordnance,  Pittsfield 

Military  Electronic  Systems,  Syracuse 
Re-entry  Systems,  Philadelphia 
Aerospace  Control  Systems,  Binghamton 

‘Responded  to  questionnaire  only  as  part  of  AIA/NSIA 
Air  Force  Liaison  Panel's  joint  response. 


Mi litary /Government : 

WPAFB : 

HQ-Air  Force  Systems 
Command  -  QA  Office 
HQ-Aeronautical  Systems 
Di vi si on /EM/PMDQ 
HQ-Air  Force  Logistics 
Command /QAA 

Office  of  the  Secretary  of 
Defense,  Asst,  for  Quality 

HQ-NASA,  Officer  of  Chief 
Engineer 
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In  some  instances,  we  received  a  response  to  the 
questionnaire  without  a  face-to-face  meeting  and  some  of  the 
meetings  did  not  result  in  a  questionnaire  response.  In 
summary,  we  conducted  14  meetings  and  received  23  responses 
to  the  questionnaire. 
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B. 3  Responses 

The  data  obtained  from  our  questionnaire  and  our  meetings 
are  summarized  below: 

B.3.1  Quality  Assurance  Rating 

The  responses  to  the  Quality  Assurance  Rating  are 
summarized  in  Figures  B-4  through  B-7.  Each  question 
will  be  discussed. 

1)  What  is  the  current  level  of  adequacy  of  MIL-Q-9858A 
in  each  of  the  Program  Life  Cycle  areas? 

The  purpose  of  this  question  was  to  get  a  better 
understanding  of  how  well  MIL-Q-9858A  was  addressing 
all  phases  of  the  Program  Life  Cycle. 

The  Needs  Analysis  data  had  already  indicated  that 
the  present  quality  programs  were  weakest  at  the 
front  end  and  the  back  end  of  the  Program  Life  Cycle 
and  strongest  in  the  middle.  The  response  to  this 
question  bears  out  the  same  data.  Note  that  a 
number  of  respondents  felt  that  M1L-Q-9858A  didn’t 
apply  to  Pre-Proposal,  Proposal  and 
Post-Production.  MIL-Q-9858A  is  thought  to  be 
moderately  adequate  for  Design  and  Development  and 
highly  adequate  in  Pre-Production  and  Production. 

2)  What  do  you  anticipate  will  be  the  level  of  adequacy 
of  MIL-Q-9858A  in  the  automated  environment  in  the 
Factory-of-the-Future  in  each  area  of  the  Program 
Life  Cycle? 

The  purpose  of  this  question  whs  to  get  feedback  on 
how  well  people  thought  MIL-Q-9858A  would  apply  in 
the  automated  environment  of  the  future. 

Once  again,  a  number  of  respondents  feel  MIL-Q-9858A 
doesn't  apply  for  Pre-Proposal,  Proposal  or 
Post-Production.  Now  the  data  is  skewed  to  the  left 
compared  to  the  responses  to  question  number  one. 
Similarly  the  data  has  moved  to  the  left  for  Design 
and  Development,  Pre-Production  and  Production 
indicating  that  there  is  a  general  feeling  that 
MIL-Q-9858A  won't  be  as  adequate  in  the 
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Factory-of-the-Future  as  it  is  now.  Note,  however,  that 
13  of  the  23  respondents  still  feel  MIL-Q-9858A  will  be 
highly  adequate  in  the  Factory-of-the-Future . 

3)  What  will  be  the  level  of  impact  of  the  increased 
usage  of  computers  and  automation  in  the 
Factory-of-the-Future  in  each  area  of  the  Program 
Life  Cycle? 

The  purpose  of  this  question  was  to  get  a  feeling 
for  how  much  change  is  coming  with  the  introduction 
of  greater  computer  usage. 

The  results  are  skewed  (very  strongly)  to  the  right 
indicating  that  the  impact  will  be  high.  The 
Pre-Proposal  area  is  a  mild  exception  with 
relatively  flat  data  indicating  not  much  change  is 
anticipated. 

4)  What  is  the  level  of  opportunity  for  improvement  in 
Quality  Assurance  in  each  area  of  the  Program  Life 
Cycle  as  a  result  of  changes  or  additions  to 
government/military  quality  assurance  standards? 

The  purpose  of  this  question  was  to  ascertain  the 
improvement  in  quality  assurance  with  new  and 
improved  standards  and  specifications. 

The  most  interesting  thing  about  this  data  is  its 
ambivalence.  Just  about  as  many  respondents  feel 
there  isn't  much  opportunity  as  feel  that  there  is. 
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Task  4  ,  4  .  4 -Summary  Of  Responses  To  Questionnaire 
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B .  3.-2 — Qual'ity  Assurance  Questionnaire 

The  second  part  o£  the  questionnaire  sought  a  written 
response  to  specific  questions  having  to  do  with  the 
current  application  of  MIL-Q-9858A  and  other  government 
specifications,  as  well  as  questions  concerning  the 
automated  environment  of  the  Factory-of -the  Future.  The 
responses  were  extremely  varied  but  we  have  tried  to 
capture  the  essence  of  the  comments  in  the  summaries  that 
follow : 

1)  How  well  does  MIL-Q-9858A  cover  your  current  Quality 
Assurance  needs?  What  are  its  weaknesses?  What  are 
its  strengths?  What  changes  would  you  recommend? 

The  purpose  of  this  question  was  to  get  specific 
comments  on  the  current  application  of  MIL-Q-9858A. 

The  overwhelming  response  was  favorable  with 
comments  such  as: 

Excellent  document 

Serves  purpose  well 

Understood 

High  comfort  level 

Flexible 

Doesn't  need  changing 

On  the  other  hand,  the  areas  for  improvement 
mentioned  most  frequently  were: 

Weak  in  design  and  development 

-  Doesn't  address  detailed  software  issues 
Needs  greater  emphasis  on  process 
control 

-  Needs  more  on  cost  performance 
measurements 

-  Needs  broader  definition  of 
"objective  evidence" 

A  concern  was  voiced  for  the  increased  risk  of 
mi  sunder standing  if  MIL-Q-9858A  were  changed. 
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2)  What  problems  have  you  encountered  as  a  result  of 
complying  with  other  gover nmen t/mi 1 i tary  Quality 
Assurance  Standards?  Identify  the  Standard  and 
discuss.  What  changes  would  you  recommend? 

The  purpose  of  this  question  was  to  learn  what 
specifications  are  now  causing  problems  and  what 
might  be  done  about  it. 

The  response  was  very  uniform  to  this  question.  The 
specifications  that  cause  a  problem,  and  thereby 
increase  the  cost,  are  the  ones  that  cover  "how 
to”.  The  contractors  want  specifications  that  tell 
clearly  “what"  the  requirements  are  but  leave  the 
"how  to"  to  the  contractors.  Most  frequently 
mentioned  were: 

-  MIL-STD-1520 ,  Noncon f orming  Material 
MIL-STD-153 5,  Procurement 
MIL-C-45662,  Calibration  System 
Requirement 

A  response  from  the  Air  Force  indicates  that  all 
three  of  these  specifications  are  being  reviewed  for 
clarification  and  improvements. 

3)  What  impediments  do  you  foresee  as  a  result  of 
current  military/government  standards  for  Quality 
Assurance  in  moving  to  the  factory  of  the  1990 's? 

How  should  government/military  standards  be  changed 
to  accommodate  increased  automation  and  computer 
usage? 

The  purpose  of  this  question  was  to  generate 
suggestions  for  areas  that  should  be  addressed  in  a 
standard  or  specification  covering  increased 
automation . 

The  opportunities  and  the  impediments  appear  to  be 
the  same  thing  only  viewed  180°  apart.  Therefore, 
we  will  only  focus  on  the  opportunities.  The  ones 
most  often  mentioned  by  the  respondents  were: 
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Reduce  the  total  number  of  separate 
speci f ications 

Prepare  for  paperless  documentation 
Provide  for  flexibility  and  don't  try  to 
standardize  on  a  specific  automation  system 

-  Provide  for  real  time  process  control 
Provide  for  product  traceability  and 

ident i f ication  of  acceptance  status  while  in 
process 

Provide  for  software  control 
Recognize  automated  product  acceptance 
Changes  will  be  gradual  —  evolution  not 
revolution 

Provide  for  new  and  fast-moving  technology 

-  Need  for  education  and  training  for  contractor 
and  Air  Force  personnel 

4)  What  do  you  perceive  as  necessary  changes  to 

government/military  Quality  Assurance  Standards  with 
the  advent  of  the  new  automated  factory  environment? 

The  purpose  of  this  question  was  very  similar  to  the 
previous  question.  We  were  looking  for 
recommendations  for  the  automated  factory. 

Some  of  the  items  mentioned  in  response  to  question 
three  were  repeated.  But,  some  new  thoughts  emerged 
such  as  : 

Need  new  definitions  for  key  words  such  as 
"design  documentation" 

More  emphasis  needs  to  be  placed  on  process 
control  with  less  on  inspection 
Need  to  break  down  software  control  into  two 
pieces : 

o  Shippable/deliverable 
o  Process  control 

Greater  need  for  flexibility 
Must  develop  new  audit  routines 
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What  has  been  your  experience  with  "Field"  Quality 
Assurance  especially  as  related  to 
Rel iab i  1 i ty /Maintainab i  l i ty  data  collection?  Are 
the  governmont/mi 1 i tary  standards  sufficient?  How 
is  this  data  collected,  distributed  and  acted  upon? 
What  recommendal ions  do  you  have  for  improving 
government/military  Quality  Assurance  Standards  in 
this  area? 

The  purpose  of  this  question  was  to  explore  further 
the  areas  of  reliability  and  maintainability.  Also, 
there  was  evidence  from  the  Needs  Analysis  that  the 
measurement  of  the  quality  assurance  program 
effectiveness  was  an  area  needing  more  attention. 

The  respondents  jumped  on  this  question  indicating  a 
universally  bad  experience  in  getting  data  back  from 
the  field  for  quality  evaluation.  One  study  showed 
the  data  itself  was  accurate  less  than  50  percent  of 
the  time.  Some  of  the  comments  were: 

Only  reliable  data  comes  from  Company  Field 
Representatives 

The  AFM  66-1  routine  is  not  suitable  for 
collection  of  quality  data 
Each  branch  of  the  service  is  different 
Problem  is  not  fault  of  MIL-Q-9858A 

The  basic  conclusion  from  the  data  we  received  is 
that  this  is  a  major  area  requiring  attention  and 
study.  Our  limited  scope  in  Task  4.4.4  doesn't 
permit  the  effort  needed  to  get  at  this  extremely 
important  quality  issue. 

What  will  be  the  impact  of  a  paperless  factory  on 
government/military  standards?  How  should  the 
standards  be  modified  to  accommodate  a  paperless 
environ ment? 

The  purpose  of  this  question  was  to  address  the 
particular  quality  assurance  issue  which  would  be 
brought  about  by  a  "paperless  factory". 
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Many  of  the  responses  were  duplicates  of  earlier 
responses  with  the  following  items  being  mentioned 
most  often: 

The  ability  to  "tailor"  the  specification  will 
be  very  important 

Need  for  flexibility  especially  in  the  software 

system  requirements 

Real  time  process  control 

-  Must  accommodate  "evolution" 

-  Automated  measurement  of  compliance 
Elimination  of  “Inspector  Stamps" 

Acceptance  of  electronic  media  for  data  and  part 
status 

7)  What  are  other  Quality  Assurance  issues  that  should 
be  addressed  in  the  development  of  new  Product 
Quality  Assurance  Standards? 

The  purpose  of  this  question  was  to  solicit  general 
comments  that  mr.y  not  have  been  made  in  response  to 
the  other  questions. 

The  comments  touched  on  many  different  areas  and 
opinions  varied  widely.  "Make  no  changes"  to 
M1L-Q-9858A  was  mentioned  as  well  as  a  statement 
'.bout  industry's  "violent  emotional  resistance"  to 
any  changes.  Items  specifically  mentioned  include: 

-  More  upfront  involvement  by  quality  assurance  is 
needed.  More  integration  with  other  functions. 
Involvement  in  pre-proposal  activities 

-  Need  new  approach  for  configuration  management 
of  software 

Consider  impact  of  multi-year  subcontract 
Consider  international  acquisitions  and 
co-producers 

-  Need  for  orientation  and  training 

Tie  incentives  to  demonstrated  quality 
performance 

Develop  stronger  corrective  action  routines 
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B.3.3  Quality  Assurance  Meeting  Summ.irior, 

The  third  area  for  gathering  information  was  our 
faco-to-face  discussions  with  key  Quality  Managers  in  the 
aerospace  industry  and  the  Department  of  Defense-  These 
meetings  were  conducted  from  January  through  May  of 
1983.  The  feedback  from  these  meetings  can  be  broken 
into  two  parts-  The  first  part  concerns  specifications 
as  they  impact  at  present  and  the  second  part  concerns 
the  impact  of  speci f ications  on  the  Factory -of-the-Future . 

B.3.3.1  The  Present 


Although  many  different  comments  were  made,  these 
four  were  the  most  universal. 

1.  Leave  MIL-Q-9858A  alone 

The  document  is  well  understood  and  doesn’t 
present  any  serious  problems  as  it  is  written. 
Most  people  felt  current  problems  were  traceable 
to  local  interpretation  of  the  document  and  were 
not  the  problem  of  the  document  itself. 

MIL-Q- 9858A  has  flexibility  but  if  there  is  a 
weakness  in  the  document  it  comes  about  from  the 
document  being  written  more  to  cover  production 
programs  than  research  and  development  programs. 

2.  "How  to"  specification  cause  problems 

The  specifications  that  support  MIL-Q-9858A  were 
frequently  considered  problems.  The  experience 
in  industry  is  that  these  are  the  ones  that 
drive  up  costs  without  a  commensurate  benefit. 
The  specifications  mentioned  most  frequently 
were : 

o  MIL-STD-1520 ,  Nonconforming  Material 

o  MIL-STD-1535,  Procurement 

It  was  felt  that  these  specifications  get  too 
specific  in  "how  to"  rather  than  in  defining 
"what"  is  required  and  allowing  the  contractor 
to  determine  how  to  meet  the  requirements. 
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3 .  Field  da  ta  1 ;;  a  ma  j or  void 

We  heard  comments,  and  all  with  deep  concern, 
from  every  visit  about  the  failure  to  receive 
reliable  data  from  field  operations.  Much  of 
the  data  that  does  come  back  to  the  contactors 
is  incorrect.  Root  causes  of  problems  are 
seldom  identified.  Most  of  the  beneficial  field 
data  is  coming  back  to  the  contractors  through 
their  Field  Representatives. 

The  most  promising  approach  recommended  to  solve 
this  problem  was  to  establish  a  few,  possibly 
three,  bases  or  field  locations  as  an 
experimental  group  and  do  whatever  is  necessary 
to  get  a  useful  flow  of  quality  information  back 
to  the  contractors  from  these  locations.  The 
feedback  from  these  three  locations  would  then 
be  analyzed  to  determine  corrective  action  needs 
for  the  programs  involved.  Following  this  an 
assessment  would  be  made  to  determine  future 
action  needed  to  assure  good  field  data  feedback. 

4.  Software  quality  assurance  is  a  major  concern 

MIL-S TD-52779  primarily  addresses  software  to  be 
shipped  and  many  people  felt  that  it  would  not 
be  practical  for  process  control  software.  In 
general,  two  major  points  were  made: 

o  Separate  guidelines  are  needed  for  shippable 
software  and  process  control  software 
o  The  software  requirements  should  be  a  part 
of  MIL-Q-9858A  or  its  replacement 

B.3.3.2  The  Future 


In  discussing  the  future,  there  was  much  discussion 
about  MIL-Q-9858A  and  whether  or  not  it  could  be 
applied  successfully  in  the  automated  environment  of 
the  Factory -of-the-Future .  Some  felt  it  would  be 
completely  inadequate  while  others  felt  it  had  all 
the  flexibility  needed  and,  with  proper 
interpretation  and  a  good  understanding  with  the 
local  AFPRO,  would  serve  the  future  factory  very 
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well.  In  fact,  one  company  has  a  highly  automated 
factory  operating  now  with  M1L-Q-9050A  and  there  are 
no  problems  with  the  document.  The  APPRO  has  access 
to  the  quality  data  through  a  computer  terminal  in 
the  Air  Force  office.  More  than  half,  and  probably 
75  percent,  of  the  Quality  Managers  we  talked  to 
felt  that  MIL-Q-9050A  would  work  satisfactorily  for 
them  in  a  more  automated  environment.  When 
discussing  the  Factory-of -the-Future ,  the  major 
issues  were: 

1.  Local  interpretation  of  the  specifications 

The  need  for  training  and  orientation  will  be 
great.  This  will  apply  to  both  the  contractors 
and  the  local  Air  Force  representatives.  In 
general,  people  felt  that  how  the  document  was 
interpreted  was  more  important  than  how  it  was 
written.  And,  finally  many  companies  said  their 
own  in-house  regulations,  practices  and 
procedures  were  already  more  stringent  than  the 
government  requirements  so  it  didn't  make  too 
much  difference  what  the  government  did  with 
their  specifications  as  long  as  the  contractors 
continued  using  their  in-house  documents. 

2.  Evolution  not  revolution 

As  the  environment  becomes  more  automated  the 
new  specifications  will  have  to  remain  flexible 
enough  to  cover  both  the  automated  and  the 
non-automa ted  factory.  Flexibility  with  some 
freedom  of  choice  becomes  even  more  important 
than  it  is  today.  The  point  was  made  that  major 
programs  can  last  for  10  to  20  years  and  the 
quality  specification  will  have  to  be  adaptable 
for  both  new  and  old  programs. 
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3.  Prevention  and  upfront  quality 

Much  was  said  during  our  discussions  about  the 
need  for  more  attention  to  quality  in  the  early 
phases  of  a  program.  This  would  involve  an 
integration  of  the  Qualicy  function  with 
Marketing,  Engineering  and  Manufacturing  in  both 
the  Pre-Proposal  and  Proposal  phase  of  the 
program  life  cycle.  This  would  involve  an  early 
understanding  of  the  customer's  desires  and 
expectations.  In  addition,  much  more  attention 
will  need  to  be  placed  on  process  control  in 
order  to  prevent  quality  problems  from 
occurring.  With  greater  attention  on  process 
control  and  prevention,  the  need  for  inspection 
activities  will  be  reduced. 

4 .  Software 

The  successful  control  of  software  is  a  vital 
key  to  the  success  of  an  automated  factory.  The 
major  points  mentioned  about  software  were: 

o  Security  of  data 
o  Configuration  control 

o  Shippable  versus  process  control  software 
o  Lack  of  traditional  "objective  evidence" 
o  Access  to  data 

5.  Built-in  quality 

With  self-adjusting  automated  equipment,  the 
very  fact  that  material  has  been  processed  may 
mean  that  it  is  in  control.  Calibration  routine 
will  need  to  be  modified  and  criteria  for 
acceptance  will  change. 
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Cone!  I_us  j_ons 

What,  then,  are  the  conclusions  from  the  surveys  that 
will  be  important  in  developing  a  new  Quality  Assurance 
Program  Management  Standard?  There  are  many  but  these 
ten  are  the  ones  we  felt  most  important  and  each  was 
considered  as  the  Quality  Assurance  Program  Management 
Standard  was  developed. 

1.  Early  involvement  of  quality  assurance  in  the 
product  life  cycle 

2„  Emphasis  on  prevention 

3.  Allow  for  evolution,  flexibility  and  tailoring 

4.  Emphasis  on  process  control 

5.  Define  "what"  not  "how  to" 

6.  Software  control 

7.  Electronic  audit  routines 

8.  Automated  measurement  of  compliance 

9.  Quality  data  and  part  status  from  electronic  media 

10.  Quality  cost  performance  measurements 

Quality  Assurance  Questionnaire 

This  section  contains  all  the  responses  to  the  Quality 
Assurance  Questionnaire. 
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B.5  Quality  Assurance  Ques  t  i  onn.i  i  re 

This  section  contains  all  the  responses  to  the  Quality 
Assurance  Questionnaire. 
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element  test  results  are  not  conclusive  and  whole  array 
data  must  be  reduced  through  Fourier  Analysis  using 
computer  techniques  that  are  not  understood  by  gov't  reps 
responsible  for  acceptance. 
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stored,  reduced,  etc.,  what  constitutes  acceptable 
evidence  of  inspection/test. 
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Guo  1 i  tv  Assurance  Rot  i  n-r, 

General  C>v»nent:  H I  L  - Q- 9Sf>SA  i*;  considered  "Quality  Program  Requirements"  for 

programs  with  hardware  as  a  product.  A  spec,  like  MIL-S-52779A 
or  Ml  L -S  TO  -  I  £>79  are  required  to  control  the  quality  requ  i  r  e~.cn  t  s 
of  software  as  a  product. 


Ques  t i onnai re  (continued) 

I  .e)  action  requirements  (paragraph  3.6  and  3.5). 

5)  o  Data  is  not  usually  used  by  the  operational  areas.  GAC  receives  the  data 
in  the  form  of  typical  reports  used  by  the  customer  and  raw  data  that  can 
be  sorted  and  tabulated  by  GAC.  The  data  is  of  a  nature  that  it  does  give 
insight  into  the  total  aircraft/system, 
o  The  data  is  used  for  identifying  problems  and  taking  corrective  action, 
for  readiness  and  provision  planning, 
o  The  reccmmendation  regarding  Gov't,  standards  are  oriented  towards 
limiting  the  human  interface  of  data  entry  or  providing  for  more 
dedicated  personnel  or  a  more  stable  work  force. 


J.  Mule' 

Supervisor  Q.E. 
Statistical/Automated  Systems 
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Pleosc  identify  position/title  of  individual  completing  this  questionnaire:  R.A.  Ra.ic 


quality  assurance  questionnaire 

Task  0  -  QA/QC/Technical  Rcnuirements/Tasks 
ICAM  Project  Priority  1105 
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Quality  Assurance  Standards  with  thi 
advent, of.  the  new  ..automated  factory 
envirtfnment? 


attachment  a  1 


(1)  M1L-Q-93SSA  adequately  highlights  QA  involvement  in  all  areas  except  Design 

Review.  One  of  its  strengths  is  that  it  allows  some  flexibility  in  implemen¬ 
tation.  One  weakness  is  that  it  is  very  broad  and  open  to  different  interpre- 
tat ions . 


(2)  MIL-STD-52779A  overlaps  MIL-Q-9S58A  dealing  with  software  rather  than  hardware. 
Examples  of  other  overlaping  documents  are  MIL-STD-45662  and  MIL-STD-930. 

(3)  Government  standards  should  focus  on  functions  of  end  product,  emphasizing 
concrol  and  adequacy.  Machine  control  rather  than  workman  control  must  be 
recognized. 


(4)  Most  of  the  changes  should  occur  in  standards  relative  to  software  Quality 

Assurance.  Software  Q.A.  must  keep  up  with  the  state-of-the-art  in  producing 
software.  Changes  should  also  occur  in  the  controls  of  processes  and 
equipment.  Answers  in  6a  and  6b  also  apply  to  this  question. 


(5)  Field  daca  is  often  incomplete.  Various  Military  Bases  do  not  report  QA  data 
in  the  same  manner. 


(6)  Changes  will  have  to  be  made  to  allow  for: 

a)  Inspector  code  numbers  in  lieu  of  stamps. 

b)  Objective  evidence  of  inspections  and  work  performed  will  be 
through  magnetic  tapes/discs. 

c)  Security  and  duplication  of  data  must  be  addressed. 

d)  Back-up  system  in  case  of  power  failure. 

(7)  More  statistical  sampling  should  be  used.  QA  involvement  in  Design  Review 
should  be  emphasized  for  complex  systems. 
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QUALITY  ASSURANCE  RATINGS 
Task  0  -  QA/QC/Tcctm  ‘ ca  1  Renuire^ents/Tasks 
I  CAM  Project  Priority  1105 


» 


a 

i 

i 


i 

* 


B-87 


^ifvent  of  the  new  automated  factory 
env  i  moment  ? 


B-88 


-  * 


9 


9 


9 


B-89 


QUM. TTY  ASSIWICE  QUESTIONNAIRE 
Task  0  -  QA/QC/Technical  Oeou  irements/Tasks 
I  CAM  Project  Priority  1105 


QUALITY  ASSl.KAfICE  Ql.'CST iQIi'IAt f>E 
Task  0  -  QA/QC/ Icclinical  Ilrnuiremcnts/Tasks 


OUALJTY  ASSimtlCE  ratings 
QA/QC/TecImical  flcniM  r  omen  ts /Tasks 
ICAM  Project  Priority  1105 


► 


I 


B 


h 


1/1 

-V 

t/i 

ro 


Ui  t/> 

c: 

•—  c  uo 

<  IDO 

r-:  £  — 

2;  ft> 
c  t 
—  >1 

r> 

i^i  c  - 


LJ  *  C. 
C-»  U 


ecu 
ex  £  a 
3  u-n 
VO  CJ  o 
t/0  ►—  V- 

<C  ^  a. 


o 

>-  02: 

—  <  iw> 

-ic- 

< 


c r 

o 


B-93 
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B • 6  Information  Gathering  Meeting  Minutes 


I CAM  PROJECT  1105 

Task  D  -  Quality  Assurance/Quality  Control 

WBS  4.4.4 _ -  Product  Assurance  Program  Standard 

Information  Gather ing  Meetings 


Location:  Vought  Corporation,  Dallas,  Texas 

Date:  8  February  1983 


Attendees:  H 

L 
R 
D 
D 
A 


L.  Henson,  V 
J.  Johnston,  V 
Neal,  V 
L.  Norwood,  V 
L.  Orth,  V 
Reingold,  V 


N.  Rogers,  V 
J.  J.  Ryan,  V 
N.  E.  Sharp,  V 
B.  R.  Shepherd,  GE 
R.  H.  Wettach,  GE 


Meeting  Minutes: 

1.  Internal  (factory)  quality  information  gathering  is  an 
area  of  need  not  directly  addressed  in  Needs  Analysis 
(actually  incorporated  within  "Early  detection/prevention 
of  quality  problems"  -  Need  #1). 

2.  Inspection  technology  has  not  kept  pace  with  manufacturing 
technology . 

3.  Needs  #4  and  5  are  one  and  the  same  (#4.  New  QA/QC 

technology  compatible  with  Factory-of-the-Future ) .  (#5. 

New  inspection  techniques  for  internal  integrity  of 
compos i tes ) . 

4.  A  system  requirement  should  be  the  elimination  of 

inspection .  ~~~  ~ 
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Task  D  -  Quality  Assurance/Quality  Control 
WBS  4.4.4  -  Product  Assurance  Program  Standard 
Information  Gathering  Meetings 


Meeting  Minutes:  (cont'd) 

5.  A  preface  to  any  new  Product  Assurance  Program  Standard 
should  indicate  that  the  current  contract  did  not  include 
a  study  of  a  potential  "To-Be"  Factory-of-the-Future 
environment  vis-a-vis  Quality  Assurance. 

6.  MIL-S-1520  and  1535  are  too  specific  -  difficult  to  comply 
with  -  specify  "how"  to  do  things,  not  just  "what"  should 
be  done. 

7.  MIL-S-52779  is  ambiguous  -  no  unified  interpretation 
available . 

8.  Field  quality  information  system  totally  inadequate. 

9.  Organization  structure  gets  in  the  way  of  early  detection 
of  QA  problems. 

10.  Software  Control 

What  type  of  document  wilL  replace  work  instructions? 

-  flow  do  you  verify  going  from  CAD  to  CAM? 

What  gets  controlled  -  a  vault  full  of  tapes? 

How  can  I  prove  agreement  with  engineering 

documentat ion? 


l 
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Task  D  -  Quality  Assurance/Quality  Control 
WBS  4.4.4  -  Product  Assurance  Program  Standard 


Information 

Gathering  Meetings 

Location : 

General  Dynamics 

,  Fort  Worth,  Texas 

Date : 

9 

February  1983 

Attendees : 

B. 

Bedell,  GD 

A.  Reingold,  V 

L. 

Clark,  GD 

B.  R.  Shepherd,  GE 

B. 

Hurren,  GD 

P.  C.  Waldrop,  GD 

P. 

Laury,  GD 

J.  P.  Watkins,  GD 

J. 

Mayben,  GD 

R.  H.  Wettach,  GE 

L. 

McNulty,  GD 

P.  Wooten,  GD 

Meeting  Minutes: 

1.  Any  new  Product  Assurance  Program  Standard  should  be 
tailorable  to  a  specific  program. 

2.  Use  consistency  in  applying  military  standards  on  large  as 
well  as  small  manufacturers. 

3.  Why  is  Technical  Skills  block  blank  in  Needs  Analysis 
matrix  vs.  QAq  node  A1  and  A5?  We  have  needs  in  both 
these  areas. 

4.  "Early  detection/prevention  of  quality  problems"  (Need  #1) 
is  " mothe rhood " . 

5.  Re  "benefits  tracking"  -  how  can  you  track  benefits  if 
your  project  doesn't  address  implementation. 

6%  reduction  in  overall  (theoretical)  program  cost  too 
conservative  -  e.g.  Systems  Requirement  #1  -  "Early 
integration  of  functional  activities"  -  if  really  done 
right  could  result  in  much  more. 


G. 
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Meeting  Minutes:  (cont'd) 

7.  9858A  "grossly  inadequate"  for  the  Factory-of-the  Future 
e.g.  paragraph  3.3  -  Work  Instructions  -  how  does  this 
apply  in  terms  of  robotically  controlled  inspections? 

8.  Cannot  effectively  utilize  9858  in  a  Flexible  Machining 
System. 

9.  Where  will  Product  Assurance  Program  Standard  go?  (Ans. 
HQ-USAF). 

10.  How  can  benefits  be  tracked  if  you  don't  implement? 

11.  An  invitation  was  extended  to  include  us  on  the  AIA/NSIA 

liaison  panel  meeting  in  Washington,  March  8,  1983  — 

opportunity  to  introduce  our  project  and  solicit 
additional  feedback. 


B-115 
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I 


I 


Loc.it  ion  : 

GeneraL  Electric 

Company , 

Utica,  New  York 

» 

Aerospace  Electronics  Syst 

em  Department 

Da  t  e : 

L 7  February  1983 

Attendees  : 

A.  Agnone,  GE 

N. 

Pelletier,  GE 

R.  Caine,  GE 

T. 

Poyer,  GE 

• 

J.  Gardner,  GE 

N. 

Scianna,  GE 

R.  Pell,  GE 

B. 

R.  Shepherd,  GE 

Meeting  Minutes: 

1.  "Buy  a  function"  -  do  not  worry  about  all  intermediate 
spec  i  f  icat  ions . 

2.  Combine  specifications  -  there  are  too  many  now. 

3.  9B58  is  as  applicable  to  the  Factory-of-the-Future  as  it 
is  to  today's  environment. 

4.  "You  should  be  aware  of  all  the  activity  in  the  AIA/NSIA 
in  terms  of  standards". 

5.  "I  do  not  see  quality  costs  anywhere  in  your  Needs 
Ana  Lysis?" 

6.  Take  the  inspection  of  composites  out  of  Needs  Analysis  - 
it's  too  specialized. 

7.  Technical  peopLe  are  too  conservative  -  hence  we  tend  to 
ove  r- i aspect . 

8.  Utica  will  send  a  consol idated  view  of  questionnaire. 


» 
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Task  D  -  Quality  Assurance/Quality  Control 
WBS  4.4.4  -  Product  Assurance  Program  Standard 
Information  Gathering  Meetings 


Location:  Joint  AIA/NSIA  Air  Force 

Liaison  Panel  Meeting 
Washington,  D.C. 

Date:  8  March  1983 

Attendees:  L.  Bauer,  AIL/Eaton  Corp. 

L.  Baugher,  Harris  Govt.  Systems 
Maj.  J.  R.  Baumgart,  HQ  USAF/RDCM 
F.  Behegan,  Westinghouse 
Col.  P.  Brown,  HQ  AFLC/QA 

A.  Davis,  Aerojet  Strategic  Propulsion 
J.  Dean,  Lear  Siegler 

I.  J.  Epstein,  0DS 

R.  A.  Fajcic,  Honeywell  Avionics 

C.  H.  Garner,  Jr.,  HQ  USAF/RDCM 
T.  Gibson,  HQ  AFSC/ALK 

R.  E.  Hannum,  Martin  Marietta 

T.  Hart,  Westinghouse 

Capt.  G.  Herd,  HQ  AFSC/ALK 

H.  Henson,  Vought 

E.  Kuhlmann,  McDonnell  Douglas 

L.  0.  Langlois,  Hughes  Aircraft 

J.  Leslie,  Texas  Instruments 

D.  L'ltalien,  HQ  USAF/RDCM 

J.  Lutz,  Fairchild  Industries 
J.  Mayben,  General  Dynamics/Ft.  Worth 

E.  Mazza.nti,  AFCMD/QA 

J.  McDermott,  Pratt  &  Whitney 

B.  Neff,  HQ  AFSC/ALK 

E.  R.  Phillips,  HQ  AFSC/ALK 
0.  D.  Samuals ,  Jr.,  HQ  AFCMC/QMA 
W.  Weitner,  AIA  Staff 
R.  H.  Wettach,  General  Electric 
L.  A.  Wilson,  Lockheed  Georgia 
B.  S.  Yolken,  TRW 
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Meeting  Minutes:  (cont'd) 

1.  GE  made  presentation  of  Task  "D" . 

2.  Panel  agreed  to  send  out  questionnaire  to  its  member  so 
that  the  panel  could  provide  an  integrated  response. 

Note:  Responses  received  from: 

-  Honeywell 

-  Lockheed-Georg ia 

-  Martin  Marietta 

-  TRW 

-  United  Technology 
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Location:  General  Electric  Company,  St.  Petersburg,  Florida 

Meeting  of  GE  Aerospace  Quality  Managers 

Date:  16  March  L983 

Attendees:  E.  M.  Uohan,  GE 

J.  N.  Canfield,  GE 
R.  Daw,  GE 
R.  Gordon,  Cl E 
J.  A.  Griffith,  GE 
E.  II.  Halpin,  GE 
D.  Hardy,  GE 
D .  Ka  tuck i ,  GE 


Meel  i  n<|  Mi  mil  os  : 

I.  Ext  rad.  from  official  moot  i  ng  minutes. 

"Bruce  Shepherd  and  Bob  Wettuch  discussed  the  work  they 
have  been  doing  for  GE  the  past  year  on  the  Quality 
Assurance  Task  supporting  the  US  Air  Force's  Integrated 
Computer-Aided  Manufacturing  (ICAM)  Program.  Working  with 
General  Dynamics,  Northrup  and  Vought,  and  using  ICAM 
modeling  techniques,  the  team  has  learned  a  great  deal 
about  how  Quality  Assurance  is  currently  being  done  in  the 
air  frame  industry.  The  purpose  of  the  next  phase  of  the 
program  is  to  obtain  additional  information  to  verify  and 
add  to  the  data  base  so  that  a  Quality  Assurance  Program 
Management  Standard  can  be  developed  for  the  automated 
environment  expected  in  the  Factory  of  the  Future.  This 
Quality  Assurance  Program  Standard  would  potentially 
supplement  or  replace  MIL-Q-9858A  and  similar 
spec i f ica t ions  in  the  automated  factory.  Bruce  and  Bob 
encouraged  us  to  complete  and  return  a  three-page 
questionnaire  to  them." 


J.  E.  Neese,  G.E. 
B.  R.  Shepherd,  GE 
R.  Simon,  GE 
R.  T.  Simpson,  GE 
R.  E.  Smith,  GE 
D.  D.  Ward,  GE 
R.  il.  Wettuch,  GE 
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Task  D  -  QuaLity  Assurance/Quality  Control 

WBS  4.4.4  -  Product  Assurance  Program  Standard 

0 

Information  Gathering  Meetings 

Meeting  Minutes:  (coat'd) 

0 

2.  Notes  relating  to  Quality  Assurance  Program  Management 

Standard : 

2.1. 

No  problems  with  9858A. 

2.2. 

AQAP  is  not  satisfactory  from  US  (and  UK)  viewpoint. 

0 

2.3. 

9858  is  broad,  flexible,  comprehensive  enough  -  if 
you  have  a  problem  it's  not  due  to  9858. 

to 

• 

4* 

Weaknesses  at  front  and  aft  end  of  life  cycle. 
Interpretations  are  made  that  take  care  of  this, 
however . 

0  ’ 

2.5. 

Field  data  collection  is  a  problem. 

2.6. 

MIL-STD-1535  vendor  certifications  —  costs  too 
much  for  value  received. 

0 

2.7. 

Deliverable  software  is  the  issue  (re:  standards, 
specs ) . 

2.8. 

Independent  evaluation  of  software  -  against  it 
(new  proposal). 

0 

2.9. 

Local  tailoring  of  9858A  is  all  that  is  required. 

0 

B-120 

0 

L  - .  *  -  _  -  -  -  -  - 

0 
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Task  D  -  Quality  Assurance/Qua l ity  Control 
WPS  4  ^  .4  _P  r  oduct  Assn  ranee  Program  Standard 
In  forma  t Lon  Gathe ring  Meetings 


Location:  WPAFB,  Dayton,  Ohio 

Date:  7  April  1983 

Attendees:  G.  Cleveland,  ASD/EN 

L.  Johnson,  AFLC/QAA 
R.  R.  Preston,  AFWAL/MLTC 
B.  R.  Shepherd,  GE 
G.  J.  Thielen,  AS  D/PM DQ 
J.  Carl,  Lt.  Col./ASD/EN 

Meeting  Minutes: 

1.  AS QC/ ANSI  Std.  3.15  1979  "Generic  Guidelines"  for  Quality 
Systems"  -  is  a  well-written  document  -  better  than 
M1L-Q-9858A  in  many  respects. 

2.  9858A  "words"  are  tailored  to  a  manual,  system.  There  is  a 
need  to  tailor  the  words  to  an  automation  environment. 

3.  9858A  is  not  a  QC  document,  it  is  a  Management  Systems 
document . 

4.  Tooling  as  media  of  inspection  must  be  addressed  in  a  new 
standards . 

5.  There  is  a  requirement  for  documentation  of  a  sensor's 
calibration  in  an  automation  environment. 

6.  How  can  a  software-controlled  machine  -  tool  be  certified? 

7.  How  do  you  verify  that  software-controlled  test  is 
adequate?  (Software  verification). 
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Task  D  -  Quality  Assurance/Quality  Control 
WBS  4.4.4  -  Product  Assurance  Program  Standard 
Information  Gathering  Meetings 


Meeting  Minutes:  (cont'd) 

8.  MIL  standards,  specifications  are  a  result  of  successful 
cases  in  the  past. 

9.  How  do  we  verify  that  a  product  truly  meets  the 
requirements  in  a  software  environment? 

10.  " Inspector-of-the-Future"  will  be  a  computer  validator. 

11.  Qualify  the  process,  not  the  product. 

12.  USAF  puts  "how  to"  on,  but  will  take  it  off  if  contractor 
can  show  a  better  way. 
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Task  D  -  Quality  Assurance/Quality  Control 
WBS  4.4.4  -  Product  Assurance  Program  Standard 
Information  Ga thering  Meetings 


Location:  Grumman  Aerospace  Corporation,  Bethpage,  New  York 


Date:  12 

April  1983 

A t  tendees :  J . 

P. 

Kingf  ield 

,  G 

J. 

J. 

Lomba  rd i , 

G 

J. 

Mule,  G 

B. 

R. 

Shepherd , 

GE 

J. 

Stigers,  G 

R. 

H. 

Wettach, 

GE 

A. 

F. 

Weyhreter 

,  G 

Meeting  Minutes: 

1.  In  F-O-F  inspection  not  necessarily  function  of 
"inspector " . 

2.  In  F-O-F  need  more  emphasis  on  prevention  than  presently 
in  9858. 

3.  [n  F-O-F  will  need  "auditing"  built  into  software- 1 i ke 
banks  do  now. 

4.  Changes  to  F-O-F  wi L 1  come  slowly  due  to  long  run  nature 
of  products.  Therefore,  need  flexibility. 

5.  In  F-O-F  need  control  over  Engineering  and  other  functions 
beside  Manufacturing. 

6.  Take  a  look  at  FAA  certification  procedures.  Will  they 
give  us  some  ideas? 

7.  For  software  need  configuration  control  and  library 
control . 
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WBS  4.4.4  -  Product  Assurance  Program  Standard 
Information  Gathering  Meetings 


Meeting  Minutes:  (cont'd) 

8.  Need  more  government  involvement  at  front  end  of  contract 
(product  Life  cycle)  instead  of  only  contact  being 
"contract  administration". 

9.  Include  customer  in  "integration  of  quality". 

10.  If  necessary,  put  special  "how  to"  instructions  in  the 
contract  and  eliminate  them  from  the  quality 
spec ifications. 
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Task  D  -  Qua lity  Assarance/Quality  Control 
WBS  4.1.4  -  Product  Assurance  Program  Standard 
Information  Gathering  Meetings 


Location:  Hughes  Aircraft  Company,  El  Segundo,  California 

Date:  13  April  1983 

Attendees:  R.  R.  Nieves,  H 

B.  R.  Shepherd,  GE 

R.  E .  Tracy,  H 
R.  H.  Wettach,  GE 


Meeting  Minutes: 

1.  John  C.  Morey,  Corporate  Manager  of  Quality  Assurance  had 
been  called  to  Washington  so  we  met  with  R.  E.  Tracy, 
Corporate  Manager  of  Reliability  Assurance. 

2.  Feedback  from  users  is  a  major  void. 

Only  system  that  works  is  Tech  Reps. 

Military  feels  contractor  is  only  one  responsible. 

Military  frequently  uses  data  without  understanding  it 

-  focuses  on  symptoms,  not  root  causes. 

3.  Military  and  contractor  should  have  mutual  objective  and 
incentives . 

Evaluate  like  hell. 

Then  become  team  members. 

4.  Need  flexibility  and  freedom. 

5.  Government  contract  writers  frequently  don't  understand 
the  "flow  down"  of  other  specifications  when  a 
specification  callout  is  made. 
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Task  D  -  Quality  Assur ance/Quality  Control 
WBS  4.4.4  -  Product  Assurance  Program  Standard 
Information  Gathering  Meetings 


Location:  McDonnell  Douglas  Corporation 

St.  Louis,  Missouri 


Date:  L4  April  1983 


Attendees:  R. 

E. 

Anderson,  MD 

R. 

D. 

Bruenger,  MD 

B . 

R. 

Shepherd,  GE 

H. 

C. 

Todt,  MD 

R. 

H. 

Wettach,  GE 

Meeting  Minutes 

;  . 

1.  To  use  MIL-Q-9858A  for  F-O-F  redefine  meaning  of  key  words 
without  changing  the  document. 

2.  Consider  9858  a  statement  of  philosophy  requiring  a 
Quality  Plan  to  support. 

3.  To  improve  field  data  problem,  focus  on  two  to  three  bases 
for  collection  of  data.  Use  Product  Support  (Tech.  Reps.) 
people.  Try  to  get  military  to  fund. 

4.  Question!  Should  Air  Force  have  direct  access  to  computer 
data  files? 

5.  Performance  is  emphasized  to  the  detriment  of 
manufacturing,  support,  etc.  Therefore,  need  more  front 
end  integration. 

6.  Software  Quality  Assurance  people  from  military  are  doing 
a  good  job  of  getting  involved. 


9 


9 


9 
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WBS  4 .4.4  -  Product  Assurance  Program  Standard 
Information  Gathering  Meetings 


Meeting  Minutes:  (cont'd) 

7.  Quality  costs  should  not  be  part  of  a  quality  requirements 
doc ument . 

8.  Requirements  of  66-1  are  designed  for  a  Maintenance 
Management  System  and  not  for  a  Quality  System. 

Therefore,  don't  try  to  use  it  for  something  it  wasn't 
intended  to  do. 
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Task  D  -  Quality  Assurance/Qua  1 i ty  Control 
WBS  4 .4 .4  -  Product  Assurance  Program  Standard 
Information  Gathering  Meetings 


Location:  Office,  Secretary  of  Defense 

Pentagon,  Washington,  DC 


Date : 


21  April  1983 


At  tendees : 


D.  Burchfield,  DoD 
T.  Diamond,  NASA 
R.  R.  L'ltalien,  USAF 
R.  R.  Preston,  USAF 
B.  R.  Shepherd,  GE 
R.  H.  Wettach,  GE 


Meeting  Minutes:  „ 

1.  Purpose  of  meeting  was  to  review  the  Task  D  effort  with 
Mr.  Del  Burchfield,  Assistant  for  Quality,  Office, 
Secretary  of  Defense. 

2.  National  Bureau  of  Standards  is  putting  together  a  very 
informative  exhibit  on  the  Factory-of- the-Future .  Will  be 
complete  in  November  and  should  be  worth  seeing. 

3.  Specifications  governing  automation  are  needed  now. 
MIL-Q-9858A  needs  to  be  slanted  in  wording  to  address  the 
automated  environment. 

4.  Areas  recommended  for  change  in  MIL-Q-9858A  include: 

Design  Control 
Design  Change  Control 
Quality  Audit 
Quality  Cost  Control 
Process  Control 
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WBS  4.4.4  -  Product  A ssurance _ Program  Standard 

Information  Gathering  Meetings 


Meeting  Minutes:  (cont'd) 

5.  Other  standards  requiring  changes: 

MIL-STD-1521  Design  Review 

6.  Philosophy  of  being  able  to  tailor  a  control  document  to 
meet  the  need  of  a  specific  program  was  reinforced. 
However,  the  contractor  should  not  be  able  to  "tailor  out 
any  part  of  the  requirements. 

7.  Leave  MIL-Q-9858A  alone,  but  start  a  new  document  which 
would  apply  only  to  people  who  have  an  automated 
environment . 

8.  Factory-of-the-Future  will  be  evolutionary  but  need  to 
establish  program  controls  now. 
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Task  D  -  Quality  Assurance/Qua  1  i ty  Control 
WBS  4.4.4  -  Product  Assurance  Program  Standard 
Information  Gathe ring  Meetings 

Location:  The  Bendix  Corporation 

Arlington,  Virginia 

Date:  21  April  1983 

Attendees:  J.  W.  Cornish,  B 

B.  R.  Shepherd,  GE 
R.  H.  Wettach,  GE 


Meeting  Minutes: 

1.  Recommend  a  review  with  Dr.  Richard  Stimson. 

2.  Real  problem  will  be  how  QAR's  (Quality  Assurance 
Representat i ves  for  government)  respond. 

3.  Does  not  like  -  1520  (MRB)  or  1535  (Vendor  Control)  as 
they  allow  the  government  to  get  too  involved. 

4.  Further  inspections  will  be  done  by  machines,  not  people. 
QA  data  will  be  automatically  collected.  QA/QC 
departments  will  be  very  small. 

5.  Predicts  QA/QC  departments  will  get  smaller.  As 
contractor's  QA/QC  departments  decrease,  what  will 
government  do?  Will  government  QA/QC  offices  also  get 
smaller? 

6.  Who  will  train  government  people? 

7.  Government  will  have  to  make  a  commitment  to  accept  less 
paperwork . 

8.  Dr.  Stimson' s  objective  with  the  C0DSIA  project  is  to 
eliminate  as  much  paperwork  and  documentation  as  possible 
that  does  not  contribute  to  product  quality. 
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WBS  4.4.4  -  Product  Assurance  Program  Standard 
Information  Gathering  Meetings 


Location : 

1) 

Texas  Instruments, 

Dallas , 

Texas 

2) 

Texas  Instruments, 

Sherman, 

Texas 

Da  te  : 

22 

April  1983 

Attendees : 

1) 

T.  Cooney,  VI 

2) 

M.  Gilliam,  TI 

A.  Herbert,  TI 

R.  Green,  TI 

J.  P.  Leslie,  TI 

W.  Grimes,  TI 

A.  Reingold,  V 

S.  Stephenson,  TI 

P.  Waldrop,  GD 

A.  Reingold,  V 

P.  Waldrop,  GD 

Meeting  Minutes: 


1.  This  presentation  was  well  attended  by  QA,  Manufacturing 
Engineering  and  I.E.  personnel.  Interest  in  the 
presentation  was  high  and  the  discussions  and  questions 
were  pertinent,  particularly  about  the  role  of  QA  in  the 
Factory  of  the  Future. 


2.  It  was  the  consensus  that  MIL-Q-9858  was  broad  enough  that 
it  in  itself  should  not  pose  a  problem  in  the  Factory  of 
the  Future,  that  is  under  a  computerized  system  of  quality 
and  process  control.  It  was  pointed  out  that  their 
inspection  records  were  stored  in  the  computer  and  that 
the  military  monitors  totally  approved  this  system. 

3.  The  prime  concern  of  the  applicability  of  MIL-Q-9858  to 
future  QA  systems  was  not  with  the  document  but  with  the 
interpretations  placed  upon  the  provisions  by  the 
government.  Their  opinion  was  that  MIL-Q-9858  was  fine  as 
it  stands,  however,  that  the  supporting  specifications  and 
standards  such  as:  MIL-STD  1520  and  MIL-STD  45662  were 
far  too  detailed  and  might  pose  a  problem  with  future 
QA/QC  procedures  and  systems. 
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Meeting  Minutes:  (cont'd) 

4.  Another  concern  was  the  security  and  validity  of 
electronic  storage  and  information  systems.  The 
possibility  that  the  security  at  such  systems  may  be 
broached  would  definitely  concern  the  government  and  that 
measures  must  be  taken  to  develop  "unbreakable"  security 
along  with  the  development  of  future  electronic  systems. 

5.  The  final  recommendation  was  that  the  detail  must  be  taken 
out  of  the  various  specifications  and  standards  pertaining 
to  the  Quality  Assurance  aspects  of  manufacturing  and  that 
visual  inspection  inferred  in  MIL-I-45208  not  be  a 
requirement  in  future  contracts.  Their  ideas  are  that 
manual  inspection  be  replaced  by  certification  of  the 
various  machines  that  produce  parts  such  that  as  long  as 
the  machine  tool  is  functioning  properly,  quality  products 
will  result.  Thus,  only  the  machine  tools  and  process 
will  be  controlled.  Along  with  such  automation,  the  need 
for  corrective  action  will  greatly  diminish  since  it  may 
well  be  cheaper  to  scrap  the  discrepant  part  and  replace 
it  with  good  parts. 
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WBS  4.4.4  -  Product  Assurance  Program  Standard 


Information  Gathering  Meetings 


Location:  Office,  Secretary  of  Defense 

Pentagon,  Washington,  DC 

Date:  6  May  L9B3 

Attendees:  F.  E.  Doherty,  DoD 

R.  R.  Preston,  USAF 

B.  R.  Shepherd,  GE 

R.  A.  Stimson,  DoD 

R.  11.  Wettach,  GE 


Meeting  Minutes: 

1.  Meeting  was  held  to  provide  a  review  of  Task  D  for 

Dr.  Richard  A.  Stimson,  Director,  Industrial  Productivity, 
Office,  Secretary  of  Defense. 

2.  Needs  Analysis  -  Early  Integration  Issue:  Quality  should 
be  the  integrator. 

3.  AIA/NSIA  is  being  asked  for  conceptual  ideas  for  changes 
to  MIL-Q-9858A,  not  merely  language  changes. 

4.  AQAP  document  is  more  restrictive,  less  tailorable  than 
9858A. 

5.  Government  should  impose  performance  requirements  only. 
Thrust  is  reducing  the  "How-to"  specifications  making  any 
new  documents  easier  to  "tailor". 

6.  Government  needs  to  reduce  restrictions  on  contractor. 


B-133 


ISP1 10513000 
18  AUGUST  L983 


I CAM  PROJECT  1105 

Task  D  -  Quality  Assurance/Qual ity  Control 
WBS  4 .4.4  -  Product  Assurance  Program  Standard 
Information  Gathering  Meetings 


Meeting  Minutes:  (cont'd) 

7.  Att i t ud ina l/Behaviora 1  issues  are  biggest  potential 
problems  to  a  Factory-of-the-Future  concept. 

8.  Nobody  is  looking  across  all  the  military  standards  and 
specifications  to  see  how  they  all  "play  together". 

9.  Results  of  the  QA/QC  fact-finding  questionnaire  were 
requested  for  OUS DRE ( AM ) I P . 
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Task  D  -  Quality  Assurance/Qua  1 i ty  Control 

WBS  4  .4  ._4 _ -  Product  Assu ranee  Program  Standard 

Inf ormation  Gathering  Meetings 


Location:  Boeing  Military  Aircraft  Company 

Wichita,  Kansas 

Date:  25  May  1983 

Attendees:  B.  Becker,  AFPRO 

M.  A.  Bein,  Boeing 
J.V.  Burleigh,  Boeing 
J.  E .  Foster,  Boeing 
C.  Hamlin,  AFPRO 
R.  Henpolsheimer ,  AFPRO 
H.  Janis,  Boeing 

R.  Kjehn,  Boeing 

H.  R.  Macy,  Boeing 


Meeting  Minutes: 

1.  MIL-Q-9858A  is  adequate.  Changes  would  not  improve  it  for 
Boeing . 

2.  Supplemental  documents  dealing  with  "How  To"  cause  the 
problems . 

3.  MIL-STD  45662  is  a  problem  with  the  requirement  to  be  able 
to  trace  products  that  may  have  been  checked/made  with  a 
gage  or  tool  found  to  be  out  of  tolerance. 

4.  Concerned  about  relation  of  MIL-S-52779A  (Software)  to: 

-  MIL-Q-9858A 

MIL-STD-1520  (Corrective  Action) 

Need  to  separate  requirements  for  shippable  software  from 
requirements  for  in-house  control  of  software. 

Requirements  are  too  expensive  for  in-house  software. 


M.  McGee,  Boeing 
L.  Meyer,  Boeing 

K.  Neaschaefer,  Boeing 
F.  Oaks,  Boeing 

R.  R.  Preston,  USAF 
J.  Ritter,  Boeing 

L.  Wagner,  Boeing 
R.  H.  Wettach,  GE 
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ing  Minutes:  (cont'd) 

Feel  need  for  earlier,  up-front  participation  from  Quality 
function  in  design  of  Reliability  Testing. 

Field  Data  Feedback  is  very  poor.  Analysis  difficult 
since  real  reason  for  removal  doesn't  get  reported.  Cause 
codes  are  not  accurate  and  not  reported  correctly. 

Actually,  system  wasn't  designed  to  feed  back  quality 
data,  but  rather  to  report  on  manpower  loading  and  spares 
replacement . 

See  rapid  changes  in  products  as  a  result  of  rapid  changes 
in  technology.  Examples  are: 

Just-in-Time  (Lot  size  of  1) 

Flexible  Manufacturing 


Will  force  faster  turn  around  on  corrective  action. 
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APPENDIX  C 

INDUSTRY  AND  DoD  REVIEW  COMMENTS 


I  i  troduct i on 

The  matrix  on  the  following  page  (Figure  C-l)  displays  a 
summary  of  the  written  comments  received  after  the  draft 
Quality  Assurance  Program  Management  Standard  was 
circulated  throughout  industry  and  various  Department  of 
Defense  agencies.  As  can  be  seen,  the  majority  of  the 
respondents  preferred  no  change  be  made  to  MIL-Q-9858A. 
Although  seven  respondents  made  no  major  recommendations 
that  fit  into  one  of  the  four  categories,  in  many  cases 
they  made  specific  recommendations  that  also  proved 
valuable  and  were  incorporated  in  our  final  Quality 
Assurance  Management  Standard.  In  all  cases,  each 
comment  was  carefully  considered  before  either  including 
or  excluding  it  from  the  final  Standard.  The  initial 
draft  of  the  Quality  Assurance  Program  Management 
Standard  has  been  included  to  provide  the  reader  with  the 
initial  version  of  the  document  addressed  by  the  review 
letters  in  Section  C.2.2.  The  authors  gratefully 
acknowledge  iLl.  those  who  took  the  time  to  review  and 
respond  to  our  work. 


SUMMARY  OF  WRITTEN  COMMENTS 
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C. 2  Draf t  Standard /Responses 

Section  C.2.1  contains  a  copy  of  the  draft  standard 
circulated  throughout  Industry  and  the  DoD.  Section 
C.2.2  contains  copies  of  the  response  made  to  the  draft. 
The  reader  may  use  these  two  sections  to  correlate 
responses  to  recommendations,  if  desired. 
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I 

C.  2 . 1  Draft  Qua  L  i  t  ^  _A  s  s  1 1  r  a  ii  c  e  _P  r  r  a  m  Management  Standard 

Tiie  following  section  contains  the  initial  draft 
Assurance  Program  Standard. 
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l.  SCOPE 

L.l  Applicability.  This  specification  shall  apply  to  all 

supplies  (including  equipments,  subsystems  and  systems) 
or  services  when  referenced  in  the  item  specification, 
contract  or  order. 

1.2  Contractual  Intent.  This  specification  requires  the 
establishment  of  a  quality  program  by  the  contractor  to 
assure  eompl iance  with  the  requirements  of  the  contract. 
The  program  and  procedures  used  to  implement  this 
specification  shall  be  developed  by  the  contractor.  The 
quality  program,  including  procedures,  processes  and 
product  shall  be  documented  in  hardcopy,  such  as  on 
paper,  transparency  or  film,  or  in  softcopy  such  as  disc, 
cartridge  or  tape  and  shall  be  subject  to  review  by  the 
government  representative.  The  contractor  must  make  the 
documentation  available  to  the  Government  Representative 
in  a  convenient  useable  form.  The  quality  program  is 
subject  to  the  disapproval  of  the  Government 
Representative  whenever  the  contractor's  procedures  do 
not  accomplish  their  objectives.  The  Government,  at  its 
option,  may  furnish  written  notice  of  the  acceptability 
of  the  contractor's  quality  program. 

1.3  Summary.  An  effective  and  economical  quality  program, 
planned  and  developed  in  consonance  with  the  contractor's 
other  administrative  and  technical  programs,  is  required 
by  this  specification.  Design  of  the  program  shall  be 
based  upon  cons ideration  of  the  technical  manufacturing 
aspects  of  production  and  related  engineering  design  and 
materials.  The  program  shall  assure  adequate  quality 
throughout  all  areas  cf  pre-contract,  contract  and 
post-contract  performance;  for  example,  proposal 
generation,  design,  development,  fabrication,  processing, 
assembly,  inspection,  test,  maintenance,  packaging, 
shipping,  storage,  site  installation  and  data  feedback 
from  end  users. 
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All  supplies  and  services  under  the  contract,  whether 
manufactured  or  performed  within  the  contractor's  plant 
or  at  any  other  source,  shall  be  controlled  at  all  points 
necessary  to  assure  conformance  to  contractual 
|  requirements-  The  program  shall  provide  for  the 

prevention  and  ready  detection  of  discrepancies  and  for 
timely  and  positive  corrective  action.  The  contractor 
shall  make  objective  evidence  of  quality  conformance 
readily  available  to  the  Government  Representative. 
Objective  evidence  may  be  in  form  of  written  material, 
i  graphs,  charts  and  pictures  either  on  paper,  film  or  as  a 

display  on  a  computer  terminal  or  video  screen. 
Instructions  and  records  for  quality  must  be  controlled. 

The  authority  and  responsibility  of  those  in  charge  of 
the  design,  development,  production,  testing,  and 
•  inspection  of  quality  shall  be  clearly  stated.  The 

program  shall  facilitate  determinations  of  the  effects  of 
quality  deficiencies  and  quality  costs  on  price. 
Facilities  and  standards  such  as  drawings,  engineering 
changes,  measuring  equipment  and  the  like  which  are 
m  necessary  for  the  creation  of  the  required  quality  shall 

|  be  effectively  managed.  The  program  shall  include  an 

effective  control  of  purchased  materials  and 
subcontracted  work.  Design,  development,  manufacturing, 
fabrication,  assembly,  testing  and  inspection  work 
conducted  within  the  contractor's  plant  shall  be 
controlled.  The  quality  program  shall  also  include 
I  effective  implementation  of  responsibilities  shared 

jointly  with  the  Government  or  related  to  Government 
functions,  such  as  control  of  Government  property  and 
Government  source  inspection. 

1.4  Relation  to  Other  Contract  Requirements.  This 
►  specification  and  any  procedure  or  document  executed  in 

implementation  thereof,  shall  be  in  addition  to  and  not 
in  derogation  of  other  contract  requirements.  The 
quality  program  requirements  set  forth  in  this 
specification  shall  be  satisfied  in  addition  to  all 
detail  requirements  contained  in  the  statement  of  work  or 
fc  in  other  parts  of  the  contract.  The  contractor  is 

responsible  for  compliance  with  all  provisions  of  the 
contract  and  for  furnishing  specified  supplies  and 
services  which  meet  all  the  requirements  of  the 
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contract.  If  any  inconsistency  exists  between  the 
contract  schedule  or  its  general  provisions  and  this 
specification,  the  contract  schedule  and  the  general 
provisions  shall  control.  The  contractor's  quality 
program  shall  be  planned  and  used  in  a  manner  to  support 
reliability  effectively. 

1.5  Relation  to  MIL-Q-9858A.  This  specification  may  be 
used  in  place  of  MIL-Q-9858A. 

1.6  Tailoring.  In  accordance  with  DOD  principles  governing 
the  application  and  tailoring  of  specifications  and 
standards  to  achieve  cost  effective  acquisitions  and  life 
cycle  ownership  of  defense  material,  this  standard  and 
its  supporting  documents  shall  be  tailored  to  the 
specific  program  and  program  phase  within  the  overall 
life  cycle.  This  tailoring  shall  encompass  the  selection 
and  application  of  methods,  tables,  sections,  individual 
paragraphs  and/or  sentences  to  be  placed  on  contract  in 
order  to  impose  only  the  essential  needs  to  preclude 
unnecessary  and  unreasonable  contract  costs. 

1.7  Flexibility.  Conti  actors  shall  be  provided  maximum 
flexibility  in  establishing  efficient  and  effective 
quality  programs  within  specified  contractual 
requirements.  A  contractor's  existing  practice  shall  bo 
considered  acceptable  when  it  can  be  shown  to  achieve  the 
basic  intent  of  this  product  assurance  document. 

2.  SUPERSEDING,  SUPPLEMENTATION  AND  ORDERING 


2.1  Applicable  Documents.  The  following  documents  of  the 

issue  in  effect  on  date  of  the  solicitation  form  a  part 
of  this  specification  to  the  extent  specified  herein, 
i 

SPECIFICATIONS 
Mi 1 i tary 

MIL-I-45208A  —  Inspection  System  Requirements 
j  MIL-C-45662A  --  Calibration  System  Requirements 

MIL-S-52779A  --  Software  Quality  Assurance  Program 
Requ irements 


> 
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2.2  Amendments  and  Revisions.  Whenever  this  specification 
is  amended  or  revised  subsequent  to  its  contractually 
effective  date,  the  contractor  may  follow  or  authorize 
his  subcon  tractor s  to  follow  the  amended  or  revised 
document  provided  no  increase  in  price  or  fee  is 
required.  The  contractor  shall  not  be  required  to  follow 
the  amended  or  revised  document  except  as  a  change  in 
contract.  If  the  contractor  elects  to  follow  the  amended 
or  revised  document,  he  shall  notify  the  Contracting 
Officer,  in  writing,  of  this  election.  When  the 
contractor  elects  to  follow  the  provisions  of  an 
amendment  or  revision,  he  must  follow  them  in  full. 

2.3  Ordering  Government  Documents.  Copies  of 
specifications,  standards  and  drawings  required  by 
contractors  in  connection  with  specific  procurements  may 
be  obtained  from  the  procuring  agency,  or  as  otherwise 
directed  by  the  Contracting  Officer. 

3.  QUALITY  PROGRAM  MANAGEMENT 

3.1  Organization.  Effective  management  for  quality  shall 
be  clearly  prescribed  by  the  contractor.  Personnel 
performing  quality  functions  shall  have  sufficient, 
well-defined  responsibility,  authority  and  the 
organizational  freedom  to  identify  and  evaluate  quality 
problems  and  to  initiate,  recommend  or  provide 
solutions.  Management  regularly  shall  review  the  status 
and  adequacy  of  the  quality  program.  The  term  "quality 
program  requirements"  as  used  herein  identifies  the 
collective  requirements  of  this  specification.  It  does 
not  mean  that  the  fulfillment  of  the  requirements  of  this 
specification  is  the  responsibility  of  any  single 
contractor's  organization,  function  or  person. 

3.2  Early  Involvement  of  Quality.  The  contractor  shall 
include  as  part  of  his  quality  assurance  program  a  plan 
for  the  early  integration  of  the  functions  of  marketing, 
design,  manufacturing,  quality  and  the  project  office. 

The  plan  shall  provide  for  a  review  of  the  customer's 
requirements  and  expectations  including  the 
identification  of  the  key  program  elements  that  are 
critical  for  program  success.  The  plan  shall  make  timely 
provisions  for  adequate  processes,  equipment,  tooling, 
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personnel  skills  and  program  controls  to  assure  meeting 
the  critical  requirements  and  to  assure  overall  product 
quality.  This  initial  planning  will  organize  the  need 
and  provide  for  research,  when  necessary,  to  update 
inspection  and  testing  techniques,  ins trumentation  and 
correlation  of  inspection  and  test  results  with 
manufacturing  methods  and  processes.  This  planning  will 
also  provide  appropriate  review  and  action  to  assure 
compatibility  of  manufacturing,  inspection,  testing  and 
documentation.  The  early  integration  of  the  functional 
activities  may  take  place  before  a  contract  is  signed  to 
clarify  how  critical  requirements  of  the  contract  are 
going  to  be  defined  and  met. 

3.3  Prevention.  The  quality  assurance  program  plan  shall 
focus  on  preventing  quality  problems.  This  may  include 
the  use  of  automated  equipment  for  test  and  inspection. 

It  may  also  provide  for  inspection  and  test  functions  to 
be  performed  by  production  operators  providing  an  audit 
plan  is  maintained  by  the  quality  assurance  organization . 

3.4  Work  Instructions.  The  quality  program  shall  assure 
that  all  work  affecting  quality  (including  such  things  as 
purchasing,  handling,  machining,  assembling,  fabricating, 
processing,  inspection,  testing,  modification, 
installation,  and  any  other  treatment  of  product, 
facilities,  standards  or  equipment  from  the  ordering  of 
materials  to  dispatch  of  shipments)  shall  be  prescribed 
in  clear  and  complete  documented  instructions  of  a  type 
appropriate  to  the  circumstances.  Work  instructions  may 
be  in  hard  copy  form  such  as  written  instructions  on 
paper  or  film,  or  in  soft  copy  form  such  as  computer 
tapes  and  discs.  Work  instructions  may  be  transmitted  to 
an  operator  or  directly  to  a  machine.  Such  instructions 
shall  provide  the  criteria  for  performing  the  work 
functions  and  they  shall  be  compatible  with  acceptance 
criteria  for  workmanship.  The  instructions  are  intended 
also  to  serve  for  supervising,  inspecting  and  managing 
work.  The  preparation  and  maintenance  of  and  compliance 
with  work  instructions  shall  be  monitored  as  a  function 
of  the  quality  program. 
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3.5  Records.  The  contractor  shall  maintain  and  use  any 
records  or  data  essential  to  the  economical  and  effective 
operation  of  his  quality  program.  These  records,  either 
as  hard  copy  or  as  soft  copy  such  as  computer  tapes  or 
discs,  shall  be  available  for  review  by  the  Government 
Representative  and  copies  of  individual  records  shall  be 
furnished  him  upon  request.  The  requirement  of 
furnishing  the  Government  Representative  copies  of 
records  shall  be  satisfied  by  furnishing  a  hard  copy  of 
the  records  or  by  furnishing  a  soft  copy,  such  as  a  tape 
or  disc,  capable  of  being  viewed  at  a  computer  terminal 
or  a  video  screen.  The  requirement  may  also  be  satisfied 
by  providing  the  Government  Representative  access  to  a 
computer  terminal  or  a  video  screen  where  the  information 
can  be  seen.  Records  are  considered  one  of  the  principal 
forms  of  objective  evidence  of  quality.  The  quality 
program  shall  assure  that  records  are  complete  and 
reliable.  Inspection  and  testing  records  shall,  as  a 
minimum,  indicate  the  nature  of  the  observations  together 
with  the  number  of  observations  made  and  the  number  and 
type  of  deficiencies  found.  Also,  records  for 
monitoring  work  performance  and  for  inspection  and 
testing  shall  indicate  the  acceptabi 1 i ty  of  work  or 
products  and  the  action  taken  in  connection  with 

def iciencies.  The  quality  program  shall  provide  for  the 
analysis  and  use  of  records  as  a  basis  for  management 
action . 

3.6  Software.  The  contractor  shall  establish  and  implement 

a  Software  Quality  Assurance  Program.  This  Program  shall 
be  documented  in  a  Software  Quality  Program  Plan  which  is 
subject  to  the  procuring  agency's  approval  for  compliance 
with  the  requirements  of  this  specification  and  the 
contract.  The  contractor  shall  ensure  that  the  personnel 
performing  the  activities  required  by  this  plan  have  the 
expertise  and  organ iza t ional  freedom  to  evaluate  software 
development  activities  and  to  initiate  or  recommend 
corrective  action. 

The  software  Quality  Program  Plan  shall  be  an  integral 
part  of  the  total  Quality  Assurance  plan.  For  items 
specified  for  delivery  on  a  contract  MIL-S-52779A  shall 
apply.  This  applies  to  either  software  alone  or  to 
software  as  a  portion  of  a  system  or  a  subsystem.  In 
this  regard  the  term  software  also  includes  firmware. 
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For  internal  software  used  for  process  control  and 
production  MIL-S-52779A  may  be  used  with  tailoring  to 
meet  specific  requirements.  At  a  minimum  the  process 
control  Software  Quality  Program  Plan  shall  address: 

o  A  Program  plan  establishing  authority  and  control 
o  Documentation  including  procedures  and  work 
instruction 

o  Design/code  reviews  including  Quality 
participation 

o  Configuration  management  including 

identification,  control  and  records  of  change 
o  Software  security 

o  Validation,  verification  and  certi f ication 
o  Audit  programs 

o  Problem  reporting  and  corrective  action 

The  Software  Quality  Program  Plans  shall  be  reviewed  on 
an  established  periodic  basis  to  assure  their 
applicability  to  the  most  current  contract  requirements. 

Corrective  Action.  The  quality  program  shall  detect 
promptly  and  correct  assignable  conditions  adverse  to 
quality.  Design,  purchasing,  manufacturing,  testing  or 
other  operations  which  could  result  in  or  have  resulted 
in  defective  supplies,  services,  facilities,  technical 
data,  standards  or  other  elements  of  contract  performance 
which  could  create  excessive  losses  or  costs  must  be 
identified  and  changed  as  a  result  of  the  quality 
program.  Corrective  action  will  extend  to  the 
performance  of  all  suppliers  and  vendors  and  will  be 
responsive  to  data  and  product  forwarded  from  users. 
Corrective  action  shall  include  as  a  minimum: 

(a)  Analysis  of  data  and  examination  of  product  scrapped 
or  reworked  to  determine  extent  and  causes; 

(b)  Analysis  of  trends  in  processes  or  performance  of 
work  to  prevent  nonconforming  product;  and 

(c)  Introduction  of  required  improvements  and 
corrections,  an  initial  review  of  the  adequacy  of 
such  measures  and  monitoring  of  the  effectiveness  of 
corrective  action  taken. 
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3.8  Costs  Related  to  Quality.  The  contractor  shall 

maintain  and  use  quality  cost  data  as  a  management 
element  of  the  quality  program.  These  data  shall  serve 
the  purpose  of  identifying  the  cost  of  both  the 
prevention  and  correction  of  nonconforming  supplies 
(e.g.,  labor  and  material  involved  in  material  spoilage 
caused  by  defective  work,  correction  of  defective  work 
and  for  quality  control  exercised  by  the  contractor  at 
subcontractor's  or  vendor's  facilities).  The  specific 
quality  cost  data  to  be  maintained  and  used  will  be 
determined  by  the  contractor.  These  data  shall,  on 
request,  be  identified  and  made  available  for  "on  site" 
review  by  the  Government  Representative. 

A  cost-benefit  analysis  will  be  required  as  support  for  a 
request  to  "tailor"  any  part  of  this  specification. 

4 .  FACILITIES  AND  STANDARDS 

4.1  Drawings,  Documentation  and  Changes.  The  contractor 

shall  assure  the  adequacy,  completeness  and  currentness 
of  design  documents  by  establishing  and  implementing  a 
con f iguration  management  plan.  Design  documents  include 
drawings  and  spec i f ica t ions  either  in  hard  copy  form  or 
in  software  form  such  as  computer  tapes  or  discs.  With 
respect  to  the  currentness  of  design  documents  and 
changes,  the  contractor  shall  assure  that  requirements 
for  the  effectivity  point  of  changes  are  met  and  that 
obsolete  design  documents  and  change  requirements  are 
removed  from  all  points  of  issue  and  use.  Some  means  of 
recording  the  effective  points  shall  be  employed  and  be 
available  to  the  Government. 

With  respect  to  design  documents,  a  procedure  shall  be 
maintained  that  shall  provide  for  the  evaluation  of  the 
engineering  adequacy  of  proposed  changes.  The  evaluation 
shall  encompass  both  the  adequacy  in  relation  to  standard 
engineering  and  design  practices  and  the  adequacy  with 
respect  to  the  design  and  purpose  of  the  product  to  which 
the  drawing  relates. 
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With  respect  to  supplemental  specifications,  process 
instructions,  production  engineering  instructions  and 
work  instructions  relating  to  a  particular  design,  the 
contractor  shall  be  responsible  for  a  review  of  their 
adequacy,  currentness  and  completeness.  The  quality 
program  must  provide  complete  coverage  of  all  information 
necessary  to  produce  an  article  in  complete  conformity 
with  requirements  of  the  design. 

The  quality  program  shall  assure  that  there  is  complete 
compliance  with  contract  requirements  for  proposing, 
approving,  and  effecting  of  engineering  changes.  The 
quality  program  shall  provide  for  monitoring  effectively 
compliance  with  contractual  engineering  changes  requiring 
approval  by  Government  design  authority.  The  quality 
program  shall  provide  for  monitoring  effectively  the 
design  documentation  changes  of  lesser  importance  not 
requiring  approval  by  Government  design  authorities. 

Delivery  of  correct  design  documentation  and  change 
information  to  the  Government  in  connection  with  data 
acquisition  shall  be  an  integral  part  of  the  quality 
program.  This  includes  full  compliance  with  contract 
requirements  concerning  rights  and  data  both  proprietary 
and  other.  The  quality  program's  responsibility  for 
drawings  and  changes  extend  to  the  drawings  and  changes 
provided  by  the  subcontractors  and  vendors  for  the 
con  tract . 

4.2  Measuring  and  Testing  Equipment.  The  contractor  shall 
provide  and  maintain  gages  and  other  measuring  and 
testing  devices  necessary  to  assure  that  supplies  conform 
to  technical  requirements.  These  devices  shall  be 
calibrated  against  certified  measurement  standards  which 
have  known  valid  relationships  to  national  standards  at 
established  periods  to  assure  continued  accuracy.  The 
objective  is  to  assure  that  inspection  and  test  equipment 
is  adjusted,  replaced  or  repaired  before  it  becomes 
inaccurate.  The  calibration  of  measuring  and  testing 
equipment  shall  be  in  conformity  with  military 
specification  MIL-C-4 5662A.  In  addition,  the  contractor 
shall  insure  the  use  of  only  such  subcontractor  and 
vendor  sources  that  depend  upon  calibration  systems  which 
effectively  control  the  accuracy  of  measuring  and  testing 
equipment . 
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4.3  Production  Tooling  Used  as  Media  of  Inspection.  When 
production  jigs,  fixtures,  tooling  masters,  templates, 
patterns  and  such  other  devices  are  used  as  media  of 
inspection,  they  shall  be  proved  for  accuracy  prior  to 
release  for  use.  These  devices  shall  be  proved  again  for 
accuracy  at  intervals  formally  established  in  a  manner  to 
cause  their  timely  adjustment,  replacement  or  repair 
prior  to  becoming  inaccurate. 

4.4  Automated  Equipment  and  Processes.  Automated  equipment 
may  be  used  as  the  media  of  inspection  when  the  equipment 
or  process  assures  conformance  to  requirements  by  the  way 
it  is  designed  and  controlled.  Completion  of  the 
automated  operation  may  be  accepted  as  objective  evidence 
of  conformance  to  requirements.  When  automated  equipment 
or  processes  are  used  as  the  measure  of  conformance  they 
shall  be  audited  on  a  timely  basis  to  assure  the  proper 
maintenance  of  control. 

4.5  Use  of  Contractor's  Inspection  Equipment.  The 
contractor's  gages,  measuring  and  testing  devices  shall 
be  made  available  for  use  by  the  Government  when  required 
to  de'.ermine  conformance  with  contract  requirements.  Ix 
conditions  warrant,  contractor's  personnel  shall  be  made 
available  for  operation  of  such  devices  and  for 
verification  of  their  accuracy  and  condition. 

4.6  Advanced  Metrology  Requirements.  The  quality  program 
shall  include  timely  identification  and  report  to  the 
Contracting  Officer  of  any  precision  measurement  need 
exceeding  the  known  state  of  the  art. 

5.  CONTROL  OF  PURCHASES 

5.1  Responsibility.  The  contractor  is  responsible  for 

assuring  that  all  supplies  and  services  procured  from  his 
suppliers  (subcontractors  and  vendors)  conform  to  the 
contract  requirements.  The  selection  of  sources  and  the 
nature  and  extent  of  control  exercised  by  the  contractor 
shall  be  dependent  upon  the  type  of  supplies,  his 
supplier's  demonstrated  capability  to  perform,  and  the 
quality  evidence  made  available.  To  assure  an  adequate 
and  economical  control  of  such  material,  the  contractor 
shall  utilize  to  the  fullest  extent  objectives  evidence 
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of  quality  furnished  by  his  suppliers.  When  the 
Government  elects  to  perform  inspection  at  a  supplier's 
plant,  such  inspection  shall  not  be  used  by  contractors 
as  evidence  of  effective  control  of  quality  by  such 
suppliers.  The  inclusion  of  a  product  on  the  Qualified 
Products  List  only  signifies  that  at  one  time  the 
manufacturer  made  a  product  which  met  specification 
requirements.  It  does  not  relieve  the  contractor  of  his 
responsibility  for  furnishing  supplies  that  meet  all 
specification  requirements  or  for  the  performance  of 
specified  inspections  and  tests  for  such  material.  The 
effectiveness  and  integrity  of  the  control  of  quality  by 
his  suppliers  shall  be  assessed  and  reviewed  by  the 
contractor  at  intervals  consistent  with  the  complexity 
and  quantity  of  product.  Inspection  of  products  upon 
delivery  to  the  contractor  shall  be  used  for  assessment 
and  review  to  the  extent  necessary  for  adequate  assurance 
of  quality.  Test  reports,  inspection  records, 
certificates  and  other  suitable  evidence  relating  to  the 
supplier's  control  of  quality  should  be  used  in  the 
contractor's  assessment  and  review.  The  contractor's 
responsibility  for  the  control  of  purchases  includes  the 
establishment  of  a  procedure  for  (1)  the  selection  ol 
qualified  suppliers,  (2)  the  transmission  of  applicable 
design  and  quality  requirements  in  the  Government 
contracts  and  associated  technical  requirements,  (3)  the 
evaluation  of  the  adequacy  of  procured  items,  and  (4) 
effective  provisions  for  early  information  feedback  and 
correction  of  nonconformances. 

5.2  Purchasing  Data.  The  contractor's  quality  program 

shall  not  be  acceptable  to  the  Government  unless  the 
contractor  requires  of  his  subcontractors  a  quality 
effort  achieving  control  of  the  quality  of  the  services 
and  supplies  which  they  provide.  The  contractor  shall 
assure  that  all  applicable  requirements  are  properly 
included  or  referenced  in  all  purchase  orders  for 
products  ultimately  to  apply  on  a  Government  contract. 
The  purchase  order  shall  contain  a  complete  description 
of  the  supplies  ordered  including,  by  statement  or 
reference,  all  applicable  requirements  for 
manufacturing,  inspections,  testing,  packaging,  and  any 
requirements  for  Government  or  contractor  inspections, 
quali f ication  or  approvals.  Technical  requirements  of 
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the  following  nature  must  bo  included  by  statement  or 
reference  as  a  part  of  the  required  clear  description: 
all  pertinent  design  documentation,  engineering  design 
change  orders,  specifications  (including  inspection 
system  or  quality  program  requirements),  reliability, 
safety,  weight,  or  other  special  requirements,  unusually 
test  or  inspection  procedures  or  equipment  and  any 
special  revision  or  model  identification.  The 
description  of  products  ordered  shall  include  a 
requirement  for  contractor  inspection  at  the 
subcontractor  or  vendor  source  when  such  action  is 
necessary  to  assure  that  the  contractor's  quality  program 
effectively  implements  the  contractor's  responsibility 
for  complete  assurance  of  product  quality.  Requirements 
shall  be  included  for  chemical  and  physical  testing  and 
recording  in  connection  with  the  purchase  of  raw 
materials  by  his  suppliers.  The  purchase  orders  must 
also  contain  a  requirement  for  such  suppliers  to  notify 
and  obtain  approval  from  the  contractor  of  changes  in 
design  of  the  products.  Necessary  instructions  should  be 
provided  when  provision  is  made  for  direct  shipment  from 
the  subcontractor  to  Government  activities. 

6 .  MANUFACTURING  CONTROL 

6.1  Materials  and  Materials  Control.  Supplier's  materials 

and  products  shall  be  subjected  to  inspection  upon 
receipt  to  the  extent  necessary  to  assure  conformance  to 
technical  requirements.  Receiving  inspection  may  be 
adjusted  upon  the  basis  of  the  quality  assurance  program, 
exercised  by  suppliers.  Evidence  of  the  suppliers’ 
satisfactory  control  of  quality  may  be  used  to  adjust  the 
amount  and  kind  of  receiving  inspection. 

The  quality  program' shall  assure  that  raw  materials  to  be 
used  in  fabrication  or  processing  of  products  conform  to 
the  applicable  physical,  chemical,  and  other  technical 
requirements.  Laboratory  testing  shall  be  employed  as 
necessary.  Suppliers  shall  be  required  by  the 
contractor's  quality  program  to  exercise  equivalent 
control  of  the  raw  materials  utilized  in  the  production 
of  the  parts  and  items  which  they  supply  to  the 
contractor.  Raw  material  awaiting  testing  must  be 
separately  identified  or  segregated  from  already  tested 


C-16 


ISP1 10  513000 
18  AUGUST  1983 


and  approved  material  but  can  be  released  for  initial 
production,  providing  that  identification  and  control  is 
maintained.  Material  tested  and  approved  must  be  kept 
identified  until  such  time  as  its  identity  is  necessarily 
obliterated  by  processing.  Controls  will  be  established 
to  prevent  the  inadvertent  use  of  material  failing  to 
pass  tests. 

6.2  Production  Processing  and  Fabrication.  The 

contractor's  quality  program  must  assure  that  all 
machining,  wiring,  batching,  shaping  and  all  basic 
production  operations  of  any  type  together  with  all 
processing  and  fabricating  of  any  type  is  accomplished 
under  controlled  conditions.  Controlled  conditions 
include  documented  work  instructions,  adequate  production 
equipment,  and  any  special  working  environment. 

Documented  work  instructions  are  considered  to  be  the 
criteria  for  much  of  the  production,  processing  and 
fabrication  work.  These  documented  work  instructions  may 
be  hard  copy  or  in  the  form  of  software  such  as  computer 
tape  or  disc.  They  may  be  transmitted  by  hard  copy,  by  a 
computer  terminal  or  directly  to  the  point  of  use.  These 
'.nstructions  are  the  criteria  for  acceptable  or 
unacceptable  "workmanship".  The  quality  program  will 
effectively  monitor  the  issuance  of  and  compliance  with 
all  of  these  work  instructions. 

Physical  examination,  measurement  or  tests  of  the 
material  or  products  processed  must  be  conducted  under 
controlled  conditions.  When  automated  equipment  and 
processes  are  used  conformance  to  requirements  may  be 
determined  without  physical  examination.  When  this 
occurs  the  automated  equipment  ar.d  processes  shall  be 
monitored  sufficiently  to  assure  control.  Both  physical 
inspection  and  process  monitoring  shall  be  provided  when 
control  is  inadequate  without  both,or  when  contract  or 
specification  requires  both. 

Inspection  and  monitoring  of  processed  material  or 
products  shall  be  accomplished  in  any  suitable  systematic 
manner  selected  by  the  contractor.  Methods  of  inspection 
and  monitoring  shall  be  corrected  any  time  their 
unsuitability  with  reasonable  evidence  is  demonstrated. 
Adherence  to  selected  methods  for  inspection  and 
monitoring  shall  be  complete  and  continuous.  Corrective 
measures  shall  be  taken  when  noncompliance  occurs. 
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Inspection  by  machine  operators,  automated  gages,  moving 
line  or  lot  sampling,  setup  or  first  piece  approval, 
production  line  inspection  station,  inspection  or  test 
department,  roving  inspectors  --  any  other  type  of 
inspection  —  shall  be  employed  in  any  combination 
desired  by  the  contractor  which  will  adequately  and 
efficiently  protect  product  quality  and  the  integrity  of 
process ing . 

Criteria  for  approval  and  rejection  shall  be  provided  for 
all  inspection  of  product  and  monitoring  of  methods, 
equipment,  and  personnel.  Means  for  identifying  rejected 
product  shall  be  provided. 

Certain  chemical,  metallurgical,  biological,  sonic, 
electronic,  and  radiological  processes  are  of  so  complex 
and  specialized  a  nature  that  much  more  than  the  ordinary 
detailing  or  work  documentation  is  required.  In  effect, 
such  processing  may  require  an  entire  work  specification 
as  contracted  with  the  normal  work  operation  instructions 
established  in  normal  plant-wide  standard  production 
control  issuances  such  as  job  operation  routing  books  and 
the  like.  For  these  special  processes,  the  contractors' 
quality  program  shall  assure  that  the  process  control 
procedures  or  specifications  are  adequate  and  that 
processing  environments  and  the  certifying,  inspection, 
authorization  and  monitoring  of  such  processes  to  the 
special  degree  necessary  for  these  ultraprecise  and 
super-complex  work  functions  are  provided. 

.3  Completed  Item  Inspection  and  Testing.  The  quality 

program  shall  assure  that  there  is  a  system  for  final 
inspection  and  test  of  completed  products.  Such  testing 
shall  provide  a  measure  of  the  overall  quality  of  the 
completed  product  and  shall  be  performed  so  that  it 
simulates,  to  a  sufficient  degree,  product  end  use  and 
functioning.  Such  simulation  frequently  involves 
appropriate  life  and  endurance  tests  and  qualification 
testing.  Final  inspection  and  testing  shall  provide  for 
reporting  to  designers  any  unusual  difficulties, 
deficiencies  or  questionable  conditions.  When 
modifications,  repairs  or  replacements  are  required  after 
final  inspection  or  testing,  there  shall  be  reinspection 
and  retesting  of  any  characteristics  affected. 
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6.4  Handling,  Storage  and  Delivery.  The  quality  program 
shall  provide  for  adequate  work  and  inspection 
instructions  for  handling,  storage,  preservation, 
packaging,  and  shipping  to  protect  the  quality  of 
products  and  prevent  damage,  loss  deterioration, 
degradation,  or  substitution  of  products.  With  respect 
to  handling,  the  quality  program  shall  require  and 
monitor  the  use  of  procedures  to  prevent  handling  damage 
to  articles.  Handling  procedures  of  this  type  include 
the  use  of  special  crates,  boxes,  containers, 
transportation  vehicles  and  any  other  facilities  for 
materials  handling.  Means  shall  be  provided  for  any 
necessary  protection  against  deterioration  or  damage  to 
products  in  storage.  Periodic  inspection  for  the 
prevention  and  results  of  such  deterioration  or  damage 
shall  be  provided.  Products  subject  to  deterioration  or 
corrosion  during  fabrication  or  interim  storage  shall  be 
cleaned  and  preserved  by  methods  which  will  protect 
against  such  deterioration  or  corrosion.  When  necessary, 
packaging  designing  and  packaging  shall  include  means  for 
accommodating  and  maintain  critical  environments  within 
packages,  e.g.,  moisture  content  levels,  gas  pressures. 
The  quality  program  shall  assure  that  when  such  packaging 
environments  must  be  maintained,  packages  are  labeled  to 
indicate  this  condition.  The  quality  program  shall 
monitor  shipping  work  to  assure  that  products  shipped  are 
accompanied  with  required  shipping  and  technical 
documents  and  that  compliance  with  Interstate  Commerce 
Commission  rules  and  other  applicable  shipping 
regulations  is  effected  to  assure  safe  arrival  and 
identification  at  destination.  In  compliance  with 
contractual  requirements,  the  quality  program  shall 
include  monitoring  provisions  for  protection  of  the 
quality  of  products  during  transit. 

6.5  Nonconforming  Material.  The  contractor  shall  establish 
and  maintain  an  effective  and  positive  system  for 
controlling  nonconforming  material,  including  procedures 
for  its  identification,  segregation,  and  disposition. 
Repair  or  rework  of  nonconforming  material  shall  be  in 
accordance  with  documented  procedures  acceptable  to  the 
Government.  The  acceptance  of  nonconforming  supplies  is 
a  prerogative  of  and  shall  be  as  prescribed  by  the 
Government  and  may  involve  a  monetary  adjustment.  All 
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noncon forming  supplies  shall  be  positively  identifier!  to 
prevent  unauthorized  use,  shipment  and  intermingling  with 
conforming  supplies.  Holding  areas  or  procedures 
mutually  agreeable  to  the  contractor  and  the  Government 
Representative  shall  be  provided  by  the  contractor.  The 
contractor  shall  make  known  to  the  Government  upon 
request  the  data  associated  with  the  costs  and  losses  in 
connection  with  scrap  and  with  rework  necessary  to 
reprocess  nonconforming  material  to  make  it  conform 
completely . 

6.6  Statistical  Quality  Control  and  Analysis.  In  addition 
to  statistical  methods  required  by  the  contract, 
statistical  planning,  analysis,  test  and  quality  control 
procedures  may  be  utilized  whenever  such  procedures  are 
suitable  to  maintain  the  required  control  of  quality. 
Sampling  plans  may  be  used  when  tests  are  destructive,  or 
when  the  records,  inherent  characteristics  of  the  product 
or  the  noncritical  application  of  the  product,  indicate 
that  a  reduction  in  inspection  or  testing  can  be  achieved 
without  jeopardizing  quality.  The  contractor  may  employ 
sampling  inspection  in  accordance  with  applicable 
military  standards  and  sampling  plans  (e.g.,  form 
MIL-STD-10  5D,  M1L-STD-414,  or  Handbooks  H  10  6,  10  7, 

108).  If  the  contractor  uses  other  sampling  plans,  they 
shall  be  subject  to  review  by  the  cognizant  Government 
Representative.  Any  sampling  plan  used  shall  provide 
valid  confidence  and  quality  levels. 

6.7  Indication  of  Inspection  Status.  The  contractor  shall 
maintain  a  positive  system  for  identifying  the 
inspection  status  of  products.  Identification  may  be 
accomplished  by  means  of  stamps,  tags,  routing  cards, 
move  tickets,  tote  box  cards,  bar  codes  or  other 
acceptable  control  devices.  Such  controls  shall  be  of  a 
design  distinctly  different  from  Government  inspection 
identi f i cat ion . 

6.8  Audits.  The  contractor  shall  maintain  a  quality  audit 
plan  to  assure  that  products  meet  contract  quality 
commitments  and  fulfill  the  customer's  quality 
requirements.  These  audits  shall  cover  the  complete 
product  life  cycle  from  the  establishment  of 
requirements,  through  design  and  production,  to  the 
post-production  review  of  operational  data. 
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7.  COORDINATED  GOVERNMENT/CONTRACTOR  ACTIONS 

7.1  Government  Inspection  at  Subcontractor  or  Vendor 

Facilities.  The  Government  reserves  the  right  to 
inspect  at  source  supplies  or  services  not  manufactured 
or  performed  with  the  contractor’s  facility.  Government 
inspection  shall  not  constitute  acceptance;  nor  shall  it 
in  any  way  replace  contractor  inspection  or  otherwise 
relieve  the  contractor  of  his  responsibility  to  furnish 
an  acceptable  end  item.  The  purpose  of  this  inspection 
is  to  assist  the  Government  Representative  at  the 
contractor’s  facility  to  determine  the  conformance  of 
supplies  or  services  with  contract  requirements.  Such 
inspection  can  only  be  requested  by  or  under 
authorization  of  the  Government  Representative.  When 
Government  inspection  is  required,  the  contractor  shall 
add  to  his  purchasing  document  the  following  statement: 

"Government  inspection  is  required  prior  to  shipment 
from  your  plant.  Upon  receipt  of  this  order, 
promptly  notify  the  Government  Representative  who 
normally  services  your  plant  so  that  appropriate 
planning  for  Government  inspection  can  be 
accomplished . . " 

When,  under  authorization  of  the  Government 
Representative,  copies  of  the  purchasing  document  are  to 
be  furnished  directly  by  the  subcontractor  or  vendor  to 
the  Government  Representative  at  his  facility  rather  than 
through  Government  channels,  the  contractor  shall  add  to 
his  purchasing  document  a  statement  substantially  as 
follows: 

"On  receipt  of  this  order,  promptly  furnish  a  copy  to 
the  Government  Representative  who  normally  services 
your  plan':,  or,  if  none,  to  the  nearest  Army,  Navy, 
Air  Force,  or  Defense  Supply  Agency  inspection 
office.  In  the  event  the  representative  or  office 
cannot  be  located,  our  purchasing  agent  should  be 
notified  immediately.” 

All  documents  and  referenced  data  for  purchases  applying 
to  a  Government  contract  shall  be  available  for  review  by 
the  Government  Representative  to  determine  compliance 
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with  the  requirements  for  the  control  of  such  purchases. 
Copies  of  purchasing  documents  required  for  Government 
purposes  shall  be  furnished  in  accordance  with  the 
instructions  of  the  Government  Representative.  The 
contractor  shall  make  available  to  the  Government 
Representative  reports,  either  as  hard  copy,  software,  or 
by  access  to  the  computer  data  base,  of  any 
nonconformance  found  on  Government  source  inspected 
supplies  and  shall  (when  requested)  require  the  supplier 
to  coordinate  with  his  Government  Representative  on 
corrective  action. 

7.2  Government  Property. 

7.2.1  Government-furnished  Material.  When  material  is 
furnished  by  the  Government,  the  contractor’s  procedures 
shall  include  at  least  the  following: 

(a)  Examination  upon  receipt,  consistent  with 
practicability  to  detect  damage  in  transit; 

(b)  Inspection  for  completeness  and  proper  type; 

(c )  Periodic  inspection  and  precautions  to  assure 
adequate  storage  conditions  and  to  guard  against 
damage  from  handling  and  deterioration  during  storage; 

(d)  Functional  testing,  either  prior  to  or  after 
installation,  or  both,  as  required  by  contract  to 
determine  satisfactory  operation; 

(e)  Identification  and  protection  from  improper  use  or 
disposition,  and 

(f)  Verification  of  quantity. 

7.2.2  Damaged  Government-furnished  Material.  The  contractor 
shall  report  to  the  Government  Representative  any 
Government-furnished  material  found  damaged, 
malfunctioning,  or  otherwise  unsuitable  for  use.  In  the 
event  of  damage  or  malfunctioning  during  or  after 
installation,  the  contractor  shall  determine  and  record 
probable  cause  and  necessity  for  withholding  material 
from  use. 
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7.2.3  Bailed  Property.  The  contractor  shall,  as  required  by 

the  terms  of  tire  Bailment  Agreement,  establish  procedures 
for  the  adequate  storage,  maintenance  and  inspection  of 
bailed  Government  property.  Records  of  all  inspections 
and  maintenance  performed  on  bailed  property  shall  be 
maintained.  These  procedures  and  records  shall  be 
subject  to  review  by  the  Government  Representative. 

8.  NOTES 

(The  following  information  is  provided  solely  for 
guidance  in  using  this  specification.  It  has  no 
contractual  s ign i f i cance .  ) 

8.1  Intended  Use.  This  specification  will  apply  to  complex 
supplies,  components,  equipments  and  systems  for  which 
the  requirements  of  MIL- I— 45208  are  inadequate  to  provide 
needed  quality  assurance.  In  such  cases,  total 
conformance  to  contract  requirements  cannot  be  obtained 
effectively  and  economically  solely  by  controlling 
inspection  and  testing.  Therefore,  it  is  essential  to 
control  work  operations  and  manufacturing  processes  as 
well  as  inspections  and  tests.  The  purpose  of  this 
control  is  not  only  to  assure  that  particular  units  of 
hardware  conform  to  contractual  requirements,  but  also  to 
assure  interface  compatibility  among  these  units  of 
hardware  when  they  collectively  comprise  major 
equipments,  subsystems  and  systems. 

8.2  Exemptions.  This  specification  will  not  be  applicable 
to  types  of  supplies  for  which  MIL-I-45208  applies.  The 
following  do  not  normally  require  the  application  of  this 
specification. 

(a)  Personal  services,  and 

(b)  Research  and  development  studies  of  a  theoretical 
nature  which  do  not  require  fabrication  of  articles. 

8.3  Order  Data.  Procurement  documents  should  specify  the 
title,  number  and  date  of  this  specification. 
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DKXQA-Q  (19  Dec  83)  1st  Ind  Af-WAL/JV^LTC 

SUBJECT:  Quality  Assurance  Management  Standard  Review  AFWAL/.\ii_  T  (. 

USA  DARCOM  Product  Assurance  5  Test  Fid  Activity,  Lexington,  KY  40511  2  7  J 

TO:  Project  Manager,  Air  Force  Wright  Aeronautical  Laboratories, 

ATTN:  AFWAL/MLTC ,  Wright -Patterson  AFB ,  OH  45433 

1.  Attachment  1  to  the  basic  letter  has  been  reviewed  in  depth.  Many  of  the 
previously  recommended  changes  to  MIL-Q-9858A  are  included  in  the  report. 

2.  Some  of  the  references  listed  on  page  3  of  the  study  are  out  of  date. 
Recommend  in  the  future  that  the  revision  letter  suffix  be  deleted  to 
preclude  a  requirement  for  update  of  documents  when  a  reference  is  revised. 

3.  The  draft  MIL-Q-9858  and  the  draft  product  assurance  program  standards  in 
attachment  1  to  the  basic  were  reviewed  with  respect  to  each  other,  the  rest 
of  the  report,  and  MIL-Q-9858A.  We  believe  those  concepts  outlined  in  the 
basic  letter  as  being  established  under  the  project  "Conceptual  Design  for 
Computer  Integrated  Manufacturing"  can  best  be  served  with  a  minimum  of 
military  standards  and  specifications. 

4.  We  recommend  that  a  MIL-Q-9858B  be  published  to  include  the  requirements 
of  your  draft  product  assurance  standard  and  revised  as  shown  in  the  changes 
on  the  enclosed  DA  Form  2028.  This  will  allow  for  both  automated  and  non- 
automated  manufacturing  processes  to  be  controlled  by  the  contractor  under 
the  same  military  specification  without  further  publications.  We  realize 
that  MIL-Q-9858  revisions  have  been  delayed  because  of  industry's  reluctance 

to  change,  and  their  voiced  opposition  to  OSD.  However,  the  need  to  make  changes 
in  MIL-Q-9858A  must  be  emphasized  rather  than  publish  additional  documents. 

5.  Your  letter  of  20  January  1984  with  the  document.  Architecture  for  Product 
Assurance,  is  being  reviewed.  It  appears  that  many  of  the  recommended 
changes  to  procedures  arc  contained  in  that  document.  I  will  let  you  know 
shortly,  by  separate  memorandum,  as  to  the  distribution  of  the  Architecture 
for  Product  Assurance  that  will  be  desirable  in  this  Command. 


6.  If  I  can  be  of  further  assistance,  please  let  me  know. 


1  Enel 
wd  end  1 
Added  1  end 
2.  DA  Forms  2028 
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Quality  Program  Requirements. 

2. 

51 

1.2 

Add  new  para  1.2. 

1.2.  Definitions: 

1.2.1.  Firmware  -  Software  instructions  or  data 
that  is  stored  in  a  fixed  or  "firm  way". 

1.2.2.  Hard  copy  -  A  printed  copy  of  machine  out¬ 
put,  a  typed  copy  or  a  manually  written  copy  in 
readable  form.  For  example,  reports,  listings, 
documents,  and  summaries. 

1.2.3.  Soft  copy  -  Data  presented  as  a  video  image, 
in  audio  or  any  other  form  which  is  not  hard  copy. 

1.2.4.  Software  -  A  set  of  computer  programs, 
procedures,  rules,  and  possibly  associated  ] 

documentation  concerned  with  the  operation  of  a 
data  processing  system,  e.g.,  compilers,  library 
routines,  manuals,  circuit  diagrams. 

3. 

51 

1.3 

5 

Renumber  present  para  1.2  through  the  end 
of  section  1. 

4. 

51 

1.3 

5 

Sentence  should  read:  Design  of  the  program 
shall  be  based  upon  consideration  of  the 
technical  and  manufacturing... 

Reason:  To  agree  with  previous  MIL-Q-9858 

5. 

52 

1.5 

Change  to  read: 

1.6.  Relation  to  MIL-I-45208.  This  specification 
contains  requirements  in  excess  of  those  in 
specification  MIL-1-45208,  Inspection  System 
Requirements,  inasmuch  as  total  conformance  to 
contract  requirements  is  obtained  best  by 
controlling  work  operations,  manufacturing 
processes  as  well  as  inspections  and  tests. 
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54  3.2 


Reason :  If  this  is  published  as  MIL-Q-9858B, 
this  would  be  appropriate. 

Change  organize  to  recognize. 

Reason:  To  correct  sentence  meaning. 

Delete  last  sentence: 

The  early  integration  of  the  functional  activities 
may  take  place  before  a  contract  is  signed.... 
Reason:  Legal  ramifications.  Although  the 
intent  of  the  sentence  is  reasonable,  it  could 
be  misconstrued  by  some  contracting  officers  or 
by  some  contractors. 

Delete  "...  such  as  computer  tapes  or  disks..." 
Reason:  Soft  copy  is  defined  in  item  1  above 
new  para  1.2. 

Change  sentence  to  read:  "...  a  software  quality 
assurance  program  for  computer  controlled  auto¬ 
mated  processes  and  other  computer  related 
processes . " 

Reason:  Because  MIL-Q-9858B  will  be  all  encom¬ 
passing,  not  all  contracts  will  require  software. 

Add  new  line  at  bottom. 

0  Analysis  of  scrap,  rework,  and  repair  costs. 
Reason:  To  reinforce  para  3.8. 

Change  to  read:  "...  in  soft  copy  form.  With 
respect. . . " 

Reason:  The  word  soft  copy  is  defined  in  new  para 
1.2  (item  1  above). 


12.  57 


TYPED  NAMC.  GRADE  OR  TITLE 


Change  the  word  standard  to  state-of-the-art. 
Reason :  What  has  been  recognized  as  standard 
engineering  and  design  practices  will  change  with 
state-of-the-ar 
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13.  58 


14.  60 


Change  to  read: 

4.4.  Automated  Equipment  and  Processes.  All 
items  manufactured  by  automated  equipment  or  pro¬ 
cesses  will  be  inspected  by  automated  methods 
throughout  the  process  by  test  equipment  which 
produces  objective  evidence  of  conformance  to 
design.  Automated  production  equipment  may  be 
used  as  the  media  of  inspection  when  the  equip¬ 
ment  or  process  assures  conformance  to  require¬ 
ments  by  the  way  it  is  designed  and  controlled. 
Completion  of  the  automated  operation  may  be 
accepted  as  objective  evidence  of  conformance 
to  requirements.  When  automated  production 
equipment  or  processes  are  used  as  the  measure 
of  conformance  they  shall  be  audited  on  a  timely 
basis  to  assure  the  proper  maintenance  of  control 

Change  word  inspections  to  inspecting. 

Reason:  Grammar. 


15.  60  5.2 


Change:  unusually  to  unusual. 
Reason:  Grammar. 


16  63  6.6 


17  63  6.6 


T  YPEO  NAME.  CMAOE  0«  TITLE 


Change  to  read:  "...without  jeopardizing  quality 
(critical  and  finite  life  items  will  normally 
be.i  inspected  100%).  The  contractor..." 

Reason:  This  expands  the  sentence  to  give 
positive  requirement. 

Change  form  to  from. 

Reason:  Wrong  word 

Change  to  read: 

Audits.  The  contractor  shall  maintain  a  quality 
audit  plan  to  assure  that  products  meet  contract 
quality  commitments.  The  audits  shall  cover  the 
complete  production  process  for  which  the 
contractor  is  responsible. 
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Reason:  As  written  in  the  draft  standard,  the 
other  requ_rements  are  not  usually  covered  by 
the  contract. 


19.  64 


7.1  21 


Change  ellipsis  to  period  so  that  the  statement 
reads  ...inspection  can  be  accomplished. 

Reason :  No  more  is  required  in  the  sentence. 


20.  65 


7.1  9 

fron 

top 


Change  software  to  soft  copy. 
Reason:  See  item  1. 


21.  66 


8.1  13 


Change  to  read:  ...hardware/software  when 

the  collectively... 


•Reference  to  line  numbers  within  the  paragraph  or 

eubperefreph. 
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January  5,  1984 
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afwaumltc 


Richard  R.  Preston,  Capt.  USAF 
Project  Manager 

Computer  Integrated  Manufacturing  Branch 

Subject:  Quality  Assurance  Management  Standard  Review 

Dear  Rick: 

The  proposed  standard  modifications  relating  to  the  use  of 
computerized  information  in  lieu  of  "hard  copy”  are  required  in  an 
automated  manufacturing  environment.  I  am  in  basic  agreement  with  what 
is  presented  in  the  document. 
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/  Jack  Draffen  /. 

'  /•  / 
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January  26,  1984 
3-7258-84-020 


AFWAL/MLTC  (513-255-6976) 

Attention:  Richard  R.  Preston,  Capt.,  USAF 
Project  Manager 

Computer  Integrated  Mfg.  Branch 


Subject:  Comments  on  Task  D/QA  Draft  Product  Assurance  Program  Standard 


Dear  Captain  Preston: 

The  following  comments  are  offered  after  completing  the  review  of  Project 

Priority  1105  Task  D/QA  Draft  Product  Assurance  Program  Standard. 

1.  The  conclusions  in  Section  3.2.4  are  useful  considerations  for  evolv¬ 
ing  an  increasingly  automated  factory  and  quality  system.  It  is  my 
opinion,  however,  that  more  Air  Force  and  industry  experience  with 
quality  needs  in  a  highly  automated  factory  is  required  before 
attempting  to  establish  those  quality  considerations  important 
enough  to  be  made  specification  requirements. 

2.  Based  on  this  opinion,  I  would  select  your  first  alternative  for  a 
near  term  change  to  MIL-Q-9858A.  The  Glossary  Addition  addresses 
the  paperless  environment  common  to  all  advanced  factory  concepts 
without  adding  quality  requirements  whose  impacts  are  not  sufficiently 
understood. 

3.  Staff  members  of  the  BMAC  Quality  Assurance  Organization  reviewed 
the  document  and  concluded  that  the  best  alternative  is  to  make  no 
changes  to  MIL-Q-9858A. 

Sincerely, 

■'ft/auut/  C  'ty*  ^  c 

Maurice  McGee 

Advanced  Technology 

Boeing  Military  Airplane  Company 

Mail  Stop  K76-86 

Phone  (316)  526-7196 
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13  January  1984 


Richard  R.  Preston,  Capt.,  USAF 
Air  Force  Wright  Aeronautical  Labs 
Wright-Patterson  AFB,  OH  45433 

Subject:  Draft  Product  Assurance  Program  Standard 


Dear  Rich: 


1.  Per  your  request,  I  have  reviewed  the  Draft  Product  Assurance 
Program  Standard  package.  My  comments  are  as  follows: 


a.  Do  not  attempt  to  change  MIL-Q-9858A.  The  document  is  well 
understood  by  most  producers  of  military  products  and  will 
continue  to  have  a  place  in  the  manufacturing  arena  for  some 
time  to  come. 


b.  The  Product  Assurance  Standard  is  the  better  way  to  go  for 
the  factory  of  the  future.  However  the  title  is  not  correct, 
nor  proper  since  it  does  not  follow  the  definition  of  Product 
Assurance  shown  on  page  6,  para.  2.2.  A  better  title  would 
be  "Quality  Assurance  Program  Standard,"  because  throughout 
the  document  the  words  "quality  program"  appear  since  it  was 
patterned  after  MIL-Q-9358A.  In  any  case,  we  should  not 
lose  the  "quality  assurance"  name  which  is  known  and  used 
international ly.  Therefore,  recommend  quality  assurance  be 
used  in  place  of  product  assurance. 

c.  The  new  standard  should  contain  the  same  titles  and  paragraph 
numbers  as  MIL-Q-9858A  since  most  of  us  in  Quality  Assurance 
remember  subjects  by  paragraph  numbers.  When  new  paragraphs 
must  be  added  such  as  para.  3.3  Prevention,  they  should  not 
be  inserted  in  existing  paragraph  numbers,  but  appear  at  the 
end  of  that  reading.  Such  as: 


i 


then  add 


3.3  Work  Instructions 

3.4  Records 

3.5  Corrective  Action 

3.6  Cost  Related  to  Quality 

3.7  Prevention 

3.8  Software 

3.9  Etc. 
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Richard  R.  Preston,  Capt.,  USAF 

Subject:  Draft  Product  Assurance  Program  Standard 

Page  2 


d.  Reference  page  53,  para.  21  Applicable  Documents:  Change 
MIL-C-45662A  to  read,  "MIL-STD-45662A" .  Title  was  changed 
10  June  1930. 

e.  Reference  page  54,  para.  3.3  Prevention:  Second  line,  delete 
word  "quality".  New  sentence  to  read,  "The  quality  assurance 
program  plan  should  focus  on  preventing  problems."  There  is 
no  such  thing  as  a  "quality  problem."  It  is  an  engineering 
problem,  a  design  problem,  a  manufacturing  problem,  etc., 
but  it  is  not  a  quality  problem.  (Reference  definition  of 
qual ity) . 

f.  Reference  pages  51,  52,  53,  para.  1.1,  1.2,  1.4,  1.5,  2.1,  2.2 
Delete  the  word  "specification"  in  these  paragraphs  and  if 
the  word  appears  elsewhere  such  as  paragraphs  8.0,  8.1,  8.2 
and  8.3,  page  66,  delete  also.  This  is  a  standard  and  not 

a  specification. 

2.  The  above  comments  were  the  results  of  a  very  rapid  review  due 
to  time  constraints.  Please  accept  these  comments  in  the  same 
good  faith  that  they  were  prepared.  Your  task  in  creating  the 
factory  of  the  future  and  it's  related  quality  checks  and  balances 
is  no  easy  job.  I  wish  to  congratulate  you  on  your  success  thus 
far.  I  am  also  very  pleased  that  you  have  the  Quality  Assurance 
Task  D  so  far  along.  If  you  recall,  it  was  my  original  request 
that  started  you  in  this  effort. 

3.  I  would  be  happy  to  participate  in  any  future  reviews  you  might 
wish  to  undertake.  If  I  can  be  of  further  assistance,  please 
let  me  know. 

Sincerely, 

Alvin  M.  Baughman 

Professional  Engineer 

State  of  California  #QU1256 
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Quality  Assurance  Management  Standard  Review  (Yr  ltr  of  19  Dec  83) 


received 
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afwal/mltc 

1.  I  appreciate  the  opportunity  to  review  the  material  you  provided.  The  views 
held  by  the  various  Military/Government  and  Industry  personnel  were  particularly 
interesting. 

2.  I've  attached  my  comments  against  the  alternative  documents  in  your  "Task  D“ 
project.  These  comments  take  into  consideration  that  MIL-Q-9858A  is  only  one 
part  of  the  overall  program  to  acquire  quality  products.  In  effect  MIL-Q-9858A 
(or  any  alternative)  is  a  single  management  system  embedded  in  many  other  manage¬ 
ment  systems.  The  influence  of  any  new  document  on  the  future  factory,  i6  tem¬ 
pered  by  the  policies  and  regulations  of  the  different  Military  Departments  and 
Agencies,  by  the  experience  and  training  of  the  work  force  (Government  and 
Industry)  and  by  its  acceptability  to  all  concerned.  Requirements  of  management 
standards  like  MIL-$-9858A  vary  depending  upon;  (1)  the  funding  provided  by  the 
Program  Office  to  establish  or  maintain  the  quality  system;  (2)  the  necessary 
adaption  by  the  contractor  to  the  products  produced  and  the  business  systems 
used;  and  (3)  interpretation  by  the  Contract  Administration  Office.  Quality 
systems  developed  under  MIL-Q-9858A  vary  from  contractor  to  contractor  today 
although  this  specification  was  published  over  20  years  ago.  Any  project  to 
develop  concepts  for  the  future  factory  should  certainly  look  at  the  overall 
acquisition  picture  as  well  as  contractual  quality  system  requirements. 

ALBERT  W.  FRY  ^  1  Atch 

Chief,  Quality  Assurance  Division  Comments 
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Comments 


MIL-Q-9858A  Glossary  Addition 

1.  Although  certainly  the  easiest  solution  to  the  problem,  this  is  also  the 
least  satisfactory.  MIL-Q-9858  Is  geared  to  a  production  effort.  It  may  suffice 
by  Itself  In  some  situations  for  the  early  program  phases  with  liberal  interpre¬ 
tation  and  a  lot  of  understanding.  With  the  glossary  changes  we  will  still  have  a 
document  ill  fitted  for  anything  other  than  late  Full  Scale  Engineering 
Development  and  subsequent  acquisition  phases.  Further  because  of  perceived 
problems  in  the  past  under  the  terms  of  MIL-Q-9858A,  a  number  of  companion  docu¬ 
ment  (e.g. ,  MIL-STD-1 520)  have  been  created.  The  glossary  approach  would  have  to 
be  extended  to  these  other  documents  as  well  as  to  maintain  any  kind  of  system 
integrity. 

2.  The  actual  definitions  in  the  glossary  are  reasonable.  One  change  in  para 
3.3.2. 1. A  is  recommended.  Replace  the  third  sentence  with  the  following:  "The 
design  may  be  documented  through  software  stored  on  a  computer  tape,  disc  or 
other  suitable  media.  " 

Rationale  -  As  written  the  sentence  implies  that  the  storage  media  is  soft¬ 
ware. 


Proposed  Additions  to  MIL-Q-9858A 

For  all  practical  purposes  this  approach  is  the  same  as  the  Glossary  Addition  and 
the  same  comments  apply. 


Product  Assurance  Program  Standard 

1.  Of  the  three  alternatives,  this  is  the  best.  It  is  questionable,  however,  to 
call  this  a  product  assurance  standard.  The  term  product  assurance"  is 

often  used  to  define  a  combined  Reliability,  Maintainability  and  Quality 
Assurance  effort.  The  proposed  standard  is  essentially  a  quality  assurance 
document. 

2.  Para  2. 1  -  Revise  MIL-C-45662A  to  MIL-STD-45662. 

Rationale  -  MIL-STD  45662A  is  obsolete. 

3.  Para  3.2  calls  for  a  plan  to  integrate  disciplines,  etc.  A  Contract  Data 
Requirements  paragraph  should  be  added  to  the  standard  to  provide  for  delivery  of 
contractor  plans  for  quality. 

Rationale  -  Decisions  early  in  the  program  influence  the  entire  course  of  the 
program.  An  efficient  vehicle  is  needed  to  document  decisions,  agreements  and 
interpretations  for  all  involved  parties. 


> 
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4.  Para  3.3  The  second  and  third  sentences  are  written  as  if  the  contractor  is 
being  given  permission  to  use  automated  equipment,  etc.  The  thrust  of  the 
paragraph  should  be  changed  to  task  the  contractor  e.  g.  "The  quality  program 
shall  plan  for  the  economical  use  of  automated  equipment  for  inspection  and 
test.  " 


5. 


Para  3.^ 


6> 


Delete  the  words  “on  site" 


from  the  fourth  sentence. 


Rationale  -  Current  contract  terms  provide  for  disclosure  of  a  wide  variety 
of  sensitive  cost  data  which  range  from  scrap  costs  which  must  be  identified  in 
proposals  to  Contractor  Performance  Reports.  The  "on-site"  proscription  unne¬ 
cessarily  restricts  effective  use  of  quality  data. 


6.  Para  4.2  Change  MIL-C-45662A  to  MIL-STD-45662 

7.  Para  4.6  Delete 


Rationale  -  If  an  actual  need  exceeds  the  state  of  the  art,  this  amount  to  a 
question  involving  impossibility  of  performance.  This  really  isn’t  a  proper  area 
for  the  contractor's  quality  program  to  address. 

8.  Add  a  new  paragraph  to  section  4  to  provide  for  participation  in  design 
reviews  as  follows. 


Design  Review.  Personnel  from  the  contractor's  engineering,  manufacturing 
and  quality  assurance  organizations  shall  conduct  a  continuing  review  of  design 
and  technical  documentation  and  changes  thereto,  to  determine  their  adequacy 
regarding  design  and  function  of  the  product,  all  necessary  information  has  been 
included,  and  requirements  are  clear  and  unambiguous  and  conform  to  standard 
engineering  practices.  Checklists  shall  be  used  in  conducting  the  review  which 
shal ’  include,  but  Is  not  limited  to,  the  following: 

a.  Use  of  materials  and  processes. 

b.  Location,  mounting,  and  interconnection  of  parts. 

c.  Failure  mode  analysis  of  the  design. 

d.  Physical  layout  with  regard  to  elimination  of  single-point  failure  modes. 

e.  Appropriateness  and  accumulation  of  tolerances. 

f.  Producabi lity  and  inspectablity  of  equipment. 

g.  Quality  history  of  similiar  designs. 

The  quality  assurance  participant  shall  ensure  that  reviews  are  documented, 
deficiencies  reported,  and  corrective  action  has  been  taken  prior  to  document 
release.  These  reviews  shall  be  used  in  subsequent  quality  planning  for  procure¬ 
ment,  fabrication,  inspection,  and  test. 
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9.  Add  a  new  paragraph  to  section  6  to  provide  for  nondestructive  inspection  as 
follows: 

Designs  shall  be  evaluated  to  identify  those  product  characteristics  which 
require  verification  by  nondestructive  inspection  methods  over  and  above  visual 
examination  e.  g.  magnetic  particle  or  radiography.  Design  margin  of  safety, 
stress,  material  fracture  and  fatigue  properties,  and  product  application  should 
be  considered  as  a  minimum  in  this  evaluation.  The  quality  program  must  assure 
that  adequate  nondestructive  inspection  methods  are  identified  and  properly 
applied  to  detect  conditions  detrimental  to  the  product.  Particular  attention 
will  be  directed  toward  processes  which  must  be  stringently  controlled,  e. g. 
welding  and  bonding,  to  assure  product  integrity. 
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REPLY  TO 

ATTN  OF  :  ENP 


DEPARTMENT  OF  THE  AIR  FORCE 

HEADQUARTERS  ARMAMENT  DIVISION  (AFSC) 
EGLIN  AIR  FORCE  BASE.  FLORIDA  32542 


JAN  25  1984 

AFWAL/MLTC 


2  0  JAN  1984 


subject :  Quality  Assurance  Management  Standard  Review  (AFWAL/MLTC  Ltr  dated 
19  Dec  1983) 

ro  AFWAL/MLTC 

1.  We  have  reviewed  the  proposed  rewrite  of  MIL-Q-9858A,  the  proposed 
draft  product  assurance  standard  and  the  related  material  as  requested. 

Our  comments  and  recommendations  are  the  following: 

a.  General  comments: 

(1)  It  appears  that  both  draft  documents  are  essentially 
MIL-Q-9858A  with  changes  made  primarily  to  permit  quality  documentation 
to  be  stored,  transmitted  and  used  in  other  than  "hardcopy"  form.  We 
concur  with  the  concept  provided  controls  to  prevent  unauthorized  changes 
are  also  required. 

(2)  The  reduction  of  paper  is  certainly  worthwhile  but  it  will 
not  substantially  improve  quality  of  products.  This  effort  does  not 
really  address  the  tremendous  potential  for  automating  statistical  process 
controls  (see  comments  below  on  paragraph  6.6). 

(3)  Several  of  our  comments  below  are  aimed  at  eliminating  even 
the  implication  that  the  government  must  or  will  approve  quality  plans. 
Contractors  are  supposed  to  be  responsible  for  quality  of  their  products 
but  they  are  virtually  all  eager  to  share  this  responsibility  with  the 
customer.  One  of  the  easiest  ways  to  share  the  responsibility  for 
quality  is  to  have  the  customer  dictate  how  you  will  do  everything. 

b.  Specific  comments: 

(1)  Page  32,  Para  3. 3. 2. 1.1.  The  proposed  definition  of 
objective  evidence  does  not  define  objective  evidence,  it  defines  the 
methods  of  presenting  it.  This  definition  should  be  concerned  primarily 
with  the  substance  of  objective  evidence.  What  is  acceptable  objective 
evidence?  The  methods  used  to  record  or  present  it  are  relatively  un¬ 
important  as  long  as  there  is  some  system  that  ensures  authenticity. 

(2)  The  proposed  "rewrite"  of  MIL-Q-9858A  appears  acceptable  if 
there  is  some  means  of  assuring  that  quality  documents  and  data  main¬ 
tained  on  computer  tapes  or  discs  is  authentic,  that  it  is  controlled 
and  that  it  can  be  traced  to  a  responsible  individual. 
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(3)  Proposed  Product  Assurance  Program  Standard 


(a)  The  title  "Quality  Assurance  Program  Standard"  would  be 
more  appropriate  since  this  document  does  not  address  areas  of  product 
assurance  other  than  quality. 

(b)  Paragraphs  1.6  and  1.7.  Recommend  deletion  of  paragraphs 
1.6  and  1.7.  It  is  understood  that  tailoring  of  standards  to  program  needs 
is  required  and  it  is  not  necessary  or  desirable  to  reemphasize  the  require¬ 
ment  in  a  contractual  document.  Likewise,  it  is  not  necessary  to  re¬ 
emphasize  the  policy  that  contractors  should  be  allowed  maximum  flexibility. 
It  is  also  inappropriate  to  require  in  a  standard  that  the  government 
consider  contractor  practices  to  be  acceptable  if  he  shows  them  to  meet  the 
"basic  intent"  of  the  proposed  document.  If  anything  should  be  emphasized 

it  should  be  the  contractor's  responsibility  for  quality  of  his  product.  The 
government  should  assume  none  of  the  responsibility  for  adequacy  of  con¬ 
tractors'  practices  whether  they  meet  "basic  intents"  or  not. 

(c)  Paragraph  2.1.  MIL-C-45662A  should  be  MIL-STD-45662 . 

(d)  Paragraph  3.2.  The  government  should  not  be  interested  in 
the  contractor's  marketing  function  or  how  he  integrates  it  with  his  other 
functions.  The  last  sentence  should  be  deleted  since  it  can  be  interpreted 
as  direction  to  perform  work  before  a  contract  actually  exists. 

(e)  Paragraph  3.3.  This  paragraph  requires  a  quality  assurance 
program  plan.  MIL-Q-9858  requires  only  that  the  quality  program  be 
documented.  This  does  not  mean  it  has  to  be  in  a  single  document  or  plan 
although  some  contractors  choose  to  do  so.  It  is  not  clear  what  automated 
inspection  and  test  or  inspection  and  test  by  production  operators  have  to 

do  with  prevention.  We  do  not  believe  the  last  two  sentences  are  needed. 
Suggest  the  first  sentence  be  changed  to  read: 

The  contractor's  quality  assurance  program  documentation 
and  planning  shall  focus  on  preventing  quality  problems. 

(f)  Paragraph  3.6.  Recommend  the  second  sentence  be  revised 

to  read: 


This  program  shall  be  documented  and  shall  be  subject  to 
review  and  disapproval  by  the  Government  Representative  whenever  the  con¬ 
tractor's  procedures  do  not  accomplish  their  objectives. 

Recommend  deletion  of  the  third  sentence  since  it  is  unnecessary  and  re¬ 
dundant.  Recommend  revision  of  the  fourth  sentence  to  read: 

The  software  quality  program  shall  be  an  integral  part  of 
the  total  quality  assurance  program. 

Recommend  deletion  of  the  word  "plan"  in  the  last  sentence. 

(g)  Paragraph  4.1.  For  consistency,  all  references  to 
"drawings"  should  be  changed  to  "design  documents". 


MIL-C-45662A  should  bo  MI1.-STD-45662 . 


(Ii)  Paragraph  4.2. 

(i)  Paragrapli  b.6.  The  tone  and  substance  of  this  paragraph 
(although  it  is  identical  to  6.6  of  MIL-Q-9858)  is  not  consistent  with  the 
modern  concept  of  statistical  quality  control.  This  paragraph  was  apparently 
written  primarily  to  allow  but  limit  sampling  for  acceptance  -  the  assumption 
apparently  being  that  100%  inspection  provides  "good"  quality.  This  para¬ 
graph  needs  a  complete  reorientation  to  encourage  or,  perferably,  require 
statistical  process  controls  with  the  objective  to  virtually  eliminate  the 
need  for  acceptance  sampling  through  automation  of  statistical  process  con¬ 
trols  . 


2.  Questions  about  the  above  comments  may  be  directed  to  Mr.  Carl  Towery 
(AUTOVON  872-8652). 


rASPEylfr' GLOVER 
irector  of  Product  Assurance 
Deputy  for  Engineering 
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Mail  Station  368 


January  16,  1984 

Richard  R.  Preston 
Captain,  USAF 
Project  Manager 

Computer  Integrated  Mfg  Branch 
Wright-Patterson  Air  Force  Base,  Ohio  45433 


Dear  Captain  Preston: 

Having  completed  a  review  of  the  material  enclosed  with  your  letter  of 

December  19,  1983,  "Q A  Management  Standard  Review",  I  would  like  to  offer  the 

following  suggestions  and  comments. 

1.  Obviously  a  great  deal  of  work  has  gone  into  the  project  to  date,  and  a  lot  of 
good  information  has  been  collected. 

2.  The  general  consensus  seems  to  be  that  M1L-Q-9858A,  with  proper 
interpretation,  is  adequate  to  encompass  factory  of  the  future. 

3.  The  suggested  revisions  (additions)  to  MIL-Q-9858A  as  presented  in  the  report 
are  acceptable  and  should  not  cause  any  problems  with  currently  accepted 
interpretations.  However,  it  is  suggested  that  any  further  consideration  of 
revision  of  M1L-Q-9858A  be  discussed  with  appropriate  members  of  the  DOD 
Quality  Council  and/or  OUSD,  since  there  is  currently  effort  underway  to 
evaluate  adequacy  of  MIL-Q-9858A  and  since  industry,  through  the  CODSIA 
organization  has  recently  submitted  comments  on  this  subject. 

4.  We  cannot  support  the  development  or  issue  of  a  new  Product  Assurance 
standard  such  as  proposed  by  this  project.  As  indicated  in  our  response,  and 
borne  out  by  the  consensus  of  other  participants,  MIL-Q-9858A  is  adequate  to 
encompass  ICAM  concepts.  A  new  standard  is  unncessary,  and  would  upset 
existing  quality  systems  which  have  evolved  for  many  years  around  MIL-Q- 
9858A,  resulting  in  significant  cost  impact,  and  little  impact  on  product 
quality. 

5.  Whatever  is  eventually  done,  must  be  done  as  a  uniform,  tri-service  program. 
Contractors  who  supply  DOD  must  not  be  forced  to  maintain  separate  quality 
systems  satisfying  individual  service  requirements. 
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DEPARTMENT  OF  THE  NAVY 

NAVAL  AIR  SYSTEMS  COMMAND 
NAVAL  AIR  SYSTEMS  COMMAND  HEADQUARTERS 

WASHINGTON  r>c  20361  IN  REPLY  REFER  TO 

RECEIVED  AIR-5162/ REM 

Ser  11 

FEBL>r,9M  XFtbr.- 

AFWAL/MLTC 

From:  Commander,  Naval  Air  Systems  Command 

To:  Department  of  the  Air  Force,  Air  Force  Wright  Aeronautical  Laboratories 

(AFSC)  Wright-Patterson  Air  Force  Base,  Ohio  45433 

Sub] :  Quality  Assurance  Management  Standard  Review 

Ref:  (a)  Air  Force  Wright  Aeronautical  Laboratories  ltr  AFWL/MLTC  (513-255- 

6976)  of  19  Dec  1983 

1.  The  Quality  Assurance  Branch  of  the  Naval  Air  Systems  Command  has  reviewed 
the  subject  document  submitted  by  reference  (a)  and  concurs  with  the  need  for 
the  project  to  determine  the  adequacy  of  present  quality  documents  for  use  in 
the  factory  of  the  future.  In  general,  the  comments  from  industry  and  the 
government  are  rational  and  constructive.  Most  feel  that  MIL-Q-9858A  will 
be  adequate  with  some  modification  owing  to  the  fact  that  the  factory  of  the 
future  will  basically  be  machine/computer  controlled  with  very  little  human 
inspection  and  less  paperwork,  i.e.,  records,  data,  etc.  We  agree  with  this 
concept,  however,  although  most  of  the  big  industry  will  probably  become  fully 
automated  many  of  the  smaller  contractors  will  continue  to  operate  as  they  do 
presently.  Therefore,  we  should  not  make  drastic  changes  to  present  quality 
documentat ion  to  accomodate  automation  and  forget  the  "little  guy".  We  feel 
that  with  the  reduction  of  human  effort  to  control  quality  in  the  factory  of 
the  future  a  great  deal  of  training  and  coordination  will  be  required  by  the 
Cove rumen t  and  industry. 
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SUBJECT:  Quality  Assurance  Management  Standard  Review 


Commander 

Air  Force  Wright  Aeronautical  Laboratories  (AFSC) 
Wright -Patterson  AFB,  OH  45433 


1.  Thank  you  for  the  opportunity  to  review  and  comment  on  the  proposed 
Quality  Assurance  Management  Standard.  I  have  asked  my  Executive  Directorate 
of  Quality  Assurance  to  conduct  the  review  for  me  and  respond  directly  to 
you.  This  interim  reply  will  be  followed  by  specific  comments  from  them. 

2.  Let  me  know  if  I  can  be  of  further  assistance  in  this  matter. 


JOSEPH  covvii  tY 
Majof  Gene-.i  ,  I'SAF 
Deputy  Pi  l-eot-'j 
i Acquis l Lion  Management' 
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SUBJECT:  Quality  Assurance  Management  Standard  Review 


TO:  Commander 

Air  Force  Wright  Aeronautical  Laboratories  (AFSC) 
ATTN:  AFWAL/ML 

Wright-Patterson  AFB,  OH  45433 


1.  References: 

a.  AFWAL/MLTC  letter,  26  Dec  83,  subject  as  above. 

b.  DLA-DD(AM)  letter,  20  Jan  84,  subject  as  above. 

2.  My  staff  has  reviewed  the  subject  standard  and  our  comments  are  at 
enclosure  1.  Since  the  proposed  standard  has  not  been  formally  submitted  to 
DoD  for  adoption,  the  comments  are  offered  in  the  same  vein  -  preliminary  and 
subject  to  additions/revisions  when  and  if  a  formal  submittal  occurs. 

3.  The  three  alternative  proposals  as  a  Quality  Assurance  Management 
Standard  indicate  that  the  AFWAL  contractor(s)  relied  on  information  from 
industry  and  Government  as  the  primary  source  for  what  should  constitute  the 
standard.  Studying  the  "factory  of  the  future"  to  determine  its  specific 
needs  in  the  area  of  product/quality  assurance  standards  and  then  comparing 
them  to  existing  documentation,  requirements,  and/or  systems  to  see  if  the 
needs  are  satisfied  would  have  been  a  more  viable  approach.  The  data,  as 
presented  in  the  reference  suggests  that  the  effort  was  started  from  a 
position  that  MIL-Q-9858A  was  Inadequate  for  future  needs  and  should  be 
revised. 

4.  The  Military  Services  and  DLA  have  not  agreed  that  there  is  a  need  for 
revising  M1L-Q-9858A.  The  Council  of  Defense  and  Space  Industries 
Associations  (CODSIA),  which  represents  Aerospace  Industries  Association, 
Electronics  Industries  Association,  National  Security  Industries  Association, 
and  Shipbuilders  Council  of  America,  made  a  recommendation  at  the  11  Aug  83 
DoD  Quality  Assurance  Council  that  MIL-Q-9858A  not  be  revised.  The  Council, 
at  its  13  Sep  83  meeting,  determined  that  a  DoD  Council  task  team  should  be 
formed  to  determine  what,  if  any,  efforts  should  be  accomplished  to  improve 
quality,  reduce  costs,  and  build  discipline  into  the  current  system.  Any 
formal  AFWAL  proposal  for  revision  to  quality  documents  should  be  submitted 
to  HQ  USAF  for  presentation  to  the  DoD  QA  Council  and  subsequently  the  task 
team. 
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DLA-QR  PAGE  2 

SUBJECT:  Quality  Assurance  Management  Standard  Review 


5.  Unnecessary  changes  to  a  system  which  is  working  only  serves  to  create 
additional  problems.  Prior  to  pursuing  efforts  on  the  subject  standard,  you 
should  evaluate  other  DoD  initiatives  to  ensure  that  economical  and  effective 
quality  programs  are  in  use  from  the  earliest  phases  of  the  life  cycle. 

FOR  THE  DIRECTOR: 

1  Enel  f 

KENDRICK  S.  CORYELL 
Colonel,  USAF 
Acting  Deputy  Executive 
Director 

Quality  Assurance 
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DLA  COMMENTS 


QUALITY  ASSURANCE  MANAGEMENT  STANDARD 


A.  General  Comments  on  the  Proposed  Alternatives: 

1.  There  are  three  proposed  alternatives: 

a .  MIL-Q-9858A  plus  a  glossary  of  definitions. 

b.  MIL-Q-9858A  with  the  definitions  incorporated. 

c.  MIL-Q-9858A  with  the  definitions  incorporated,  some  minor 
rewording,  additional  paragraphs  on  Tailoring,  Flexibility,  Prevention, 
Software,  Automated  Equipment  and  Processes,  Audits,  and  a  name  change  to 
Product  Assurance  Program  Standard. 

2.  Data  presented  in  the  study  does  not  support  recommending  changes  to 
M1L-Q-9858A.  Most  of  the  respondents  felt  the  document  is  adequate  for  the 
factory  of  the  future. 

3.  No  substantive  difference  exists  between  the  three  alternatives. 

They  are  essentially  aimed  at  incorporating  software  terminology/software 
products  into  various  definitions  and  ensuring  the  involvement  of  various 
functions  within  a  company  during  the  earliest  stages  of  a  program  or 
product.  Both  are  areas  with  a  potential  for  improvement.  This  has  been 
recognized  and  is  being  pursued  within  DoD  through  efforts  such  as: 

a.  A  draft  DoD  (>iality  Design  Review  Handbook  is  currently  being 
tested  by  the  Military  Services.  If  adopted  for  use,  this  document  will 
ensure  early  involvement  of  quality,  both  Government  and  industry,  in  all 
phases  of  the  Systems  Acquisition  Cycle. 

b.  Development  of  a  new  software  quality  assurance  document.  This 
joint  Service  document  will  satisfy  most,  if  not  all,  proposed  changes  to 
MIL-Q-9838A  in  the  area  of  software,  deliverable  as  well  as  nondeliverable. 

B.  Specific  Comments: 

1.  Reference  Task  D  Report,  paragraph  3.3.2. 1,  page  32-34. 

a.  The  definitions  should  be  revised  to  reflect  that  "reviewing 
documentation  on  a  CRT  screen  will,  in  some  circumstances,  satisfy  the 
requirement  for  providing  copies,  making  available,  etc.,  documentation  to 
the  Government  Representative."  The  requirement  for  hard  copy,  however, 
will  still  exist  in  certain  circumstances,  such  as  a  disagreement,  dispute, 
request  for  data  from  contracting  officer  or  technical  activity,  etc. 
Recommend  that  the  requirement  for  hard  copy  be  left  as  an  option  if  this 
approach  i6  selected. 


b.  If  sof tware/sof tware  products  arc  to  be  used  then  a  requirement  tor 
software/Software  System  Validation  and  System  Integrity  and  Protection  must 
also  be  created.  This  section  of  the  report  does  not  propose  such  controls. 

2.  Reference  Task  D  Report,  paragraph  3. 3. 2. 2,  pages  35-49. 
a.  Same  as  comments  in  paragraph  la  and  b  above. 

3.  Reference  Task  D  Report,  paragraph  3. 3. 2. 3,  pages  50-66. 

a.  Same  as  comments  in  paragraph  la  and  b  above. 

b.  Change  the  name  of  the  standard  to  Quality  Assurance  Program 
Standard  or  identify  it  for  what  it  really  is  -  a  revision  to  MIL-Q-9858A. 

There  are  no  requirements  in  the  standard  which  encompass  such  areas  of  product 
assurance  as  reliability,  maintainability,  and  availability.  The  name  should 
not  reflect  product  assurance  unless  it  covers  the  full  spectrum  of  Product 
Assurance. 
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SUBJECT  : 


Quality  Assurance  Management  Standard  Review  (Your  letter  19  Dec  83,  same 
subject ) 


n 


The  following  is  a  list  of  comments  on  subject  draft  QA  Management 
Standard : 


a.  MIL-Q-9858A  Glossary  Addition,  Paragraph  3.3.2. 1»5,  Work 
Instructions . 


Work  instructions  should  not  be  considered  a  form  of  objective 
P.  evidence. 

Rationale :  The  MIL-STD-109B  definition  of  objective  quality  evi¬ 

dence  reads,  “Any  statement  of  fact,  either  quantitative  or  qualitative, 
pertaininq  to  the  quality  of  a  product  or  service  based  on  observations, 

^  measurements,  or  tests  which  can  be  verified."  Work  instructions  don't  fit 

0  this  definition. 

MIL-Q-9858A  Rewrite,  Paragraph  2.1. 

Change  MIL-C-45662  to  MIL-STD-45662. 

Product  Assurance  Standard. 

Paragraph  1,6,  Tailoring. 

Recommend  that  this  paragraph  be  deleted. 

Rationale :  A  document  that  is  self  tailorable  is  preferred. 

MIL-Q-9858A  is  considered  by  most  QA  professionals  to  be  self  tailorable. 

By  this  we  mean,  tailorable  in  its  application  by  the  contractor  and  its 
enforcement  by  the  Government.  (i.e.,  requirements  only  apply  if  they  fit 
the  product  or  service  being  procured  and  the  contractors  manufacturing 
.  processes.)  This  standard  should  be  self  tailorable,  in  the  same  sense, 

M  as  written.  Tailoring  may  still  be  done  but  only  in  exceptional  cases. 


Paragraph  1.7,  Flexibility. 

Recommend  that  this  paragraph  be  deleted. 

•  Rationale :  We  believe  that  contractors  are  provided  flexibility 

with  MIL-Q-9858A.  Under  current  practices  the  proof  of  conformance  to 
contract  QA  requirements  rests  with  the  contractor.  This  is  the  only 


practical  way  for  the  Government  to  operate.  The  alternative  to  this  would 
require  the  Government  to  prove  an  impact  on  product  in  order  to  get 
contractors  to  change  their  QA  systems.  This  usually  cannot  be  done  until 
it  is  to  late.  (i.e.,  The  product  is  already  nonconforming.)  This 
paragraph  weakens  the  current  practice  of  requiring  contractor  proof  of 
compliance. 

Paragraph  2.1,  Applicable  Documents. 

Change  MIL-C-45662A  to  MIL-STD-45662. 

Paragraph  3.3,  Prevention. 

Recommend  this  paragraph  be  rewritten. 

Rationale:  What  you  describe  in  this  paragraph  is  not  prevention, 
it  is  merely  another  form  of  detection.  Prevention  includes  efforts 
intending  to  show,  motivate  or  teach  how  to  do  the  job  right  the  first 
time.  Included,  for  example  would  be  efforts  such  as  preparing  and 
reviewing  work  instructions,  establishing  process  controls  and  training  or 
motivating  workers.  Controlling  automated  equipment  in  a  fashion  to  elimi¬ 
nate  or  reduce  nonconforming  material  is  prevention,  however,  any  form  of 
inspection  or  test  is  not. 

Paragraph  3.6,  Software. 

Recommend  this  paragraph  be  rewritten  as  follows: 

"The  contractor  shall  establish  and  maintain  a  Software  Quality 
Assurance  (SQA>  Program  that  is  consistent  with  MIL-S-52 779A,  Software 
Quality  Assurance  Program  Requirements.  This  program  shall  be  integral 
with  other  quality  assurance  programs,  and  shall  be  documented  in  a  plan 
that  may  stand  alone,  or  be  incorporated  into  other,  related  plans,  e.g., 
the  CPDP. 

This  SQA  Program  applies  to  deliverable,  non-deliverable,  and 
support  software  whether  purchased  as  part  of  an  overall  system  or  as 
stand-alone  software. 

In  all  instances  tailoring  of  MIL-S-52 779A  will  be  considered  to 
ensure  appropriate,  cost-effective,  controls  appropriate  to  the  specific 
purchase.  All  such  tailoring  will  be  reflected  in  the  contract  statement 
of  work." 


Rationale:  MIL-S-52 779A,  as  currently  written,  will  handle  non- 
deliverable,  support  software.  The  only  caveat  is  that  it  must  be  tailored 
for  the  specific  application.  This  is  in  accordance  with  soon  to  be 
published  guidelines  from  AFSC/ALK. 


Paragraph  4.2,  Measuring  and  Testing  Equipment. 

Change  MIL-C-45662A  to  MIL-STD-45662. 

Paragraph  6.8,  Audits. 

The  words  "to  the  post-production  review  of  operational  data"  need 
to  be  clarified.  It  is  unclear  how  a  contractor's  audit  system  would  cover 
this  area.  Without  major  changes  to  the  DOD  contracting  and  field  data 
collection  systems,  the  contractors  role  is  limited  to  responding  to  field 
complaints  and  analyzing  data  provided  by  the  the  user. 
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DEPARTMENT  OF  THE  AIR  FORCE 

HEADQUARTERS  SPACE  DIVISION  (AFSC) 

LOS  ANQEiES  AIR  FORCE  STATION.  PO  BOX  $2960,  WORLDWAY  POSTAL  CENTER 
LOS  ANOELES.  CA  90009 


FEB  1  v.m 

AFVVAL/MLTC 
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subject  Quality  Assurance  Management  Standard  Review  (Your  Ltr,  19  Dec  83) 


to  AFWAL/MLTC 

We  have  reviewed  the  draft  Quality  Assurance  Standards.  In  paragraph  3.6  of 
the  draft  Product  Assurance  Program  Standard  we  suggest  that  the  requirement 
for  a  Software  Quality  Assurance  Program  Plan  be  deleted  or  made  optional. 
The  wording  of  this  paragraph  3.6  implies  that  this  plan  somehow  becomes  a 
compliance  document.  We  recommend  that  this  paragraph  state  that  if  a  plan 
is  necessary  that  it  clearly  not  take  precedence  over  the  software  quality 
requirements  (i.e.  MIL-S-52779A)  in  the  Statement  of  Work  of  the  contract. 
This  is  consistent  with  AFSC  policy  and  may  avoid  the  problems  we  have 
experienced  when  plans  are  perceived  as  being  compliance  documents  rather 
than  information  data  items. 

s' 

C 

ALDO 

Chief,  Product  Assurance  Division 
Directorate  of  Productivity 


DEPARTMENT  OF  THE  AIR  FORCE 

HEADQUARTERS  ARNOLD  ENGINEERING  DEVELOPMENT  CENTER  (AFSC) 
ARNOLD  AIR  FORCE  STATION  TENNESSEE  3738V 
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received 

JAN  r1  i  1 7-'  i 
^VVAL/MLTC 


AFWAL/MLTC 


1.  The  Quality  Directorate  at  Arnold  Air  Force  Station,  TN  wishes  to 
thank  AFWAL  for  the  opportunity  to  review  and  comment  on  the  subject 
standard. 

2.  The  Quality  Directorate  finds  the  proposed  additions  to  M1L-Q-9858A 
beneficial  in  reducing  the  implicit  or  commonly  interpreted  requirements 
of  the  current  version  of  MIL-STD-9858A. 

3.  The  proposed  Product  Assurance  Program  Standard  (PAPS)  is  lengthy, 
but  very  explicit  in  defining  the  requirements  of  a  Quality  Assurance 
program.  The  additions  of  the  early  involvement  paragraph,  software 
paragraph,  automatic  equipment  and  processes  paragraph,  as  well  as  the 
audit  paragraphs,  are  much  needed  additions  to  quality  program  management 
directives. 

4.  There  are  two  recommendations  for  consideration:  (1)  throughout 
the  PAPS  there  is  reference  to  specific  documents  and  revision  numbers, 
(i.e.,  MIL-I-45208A,  MIL-S-52779A,  etc...).  Deletion  of  revision  identi¬ 
fiers  would  reduce  the  need  for  administrative  updates.  Second,  paragraph 
1.5  Relation  to  MIL-Q-9858A,  should  be  explicit  in  stating  at  no  time  would 

on  the  same  contract. 


both  MIL-Q-9858  and  PAPS  be  used 


'L.  I^MITCHELL,  JR.,j£jtr,  USAF 
Di lector  of  Quality  ‘Assurance 
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OH  K  I  Ol  THE  UNDER  SECRETARY  OF  DE  FENSE 

WASHINGTON  l)(  ?OI«l 


80  JAN  1934 


RE St ARCH  ANO 
ENGINEERING 


Captain  Richard  R.  Preston 

Air  Force  Wright  Aeronautical  Laboratories 

Code:  AFWAL/MLTC, 

Wright-Patterson  AFB,  OH  45433 

Dear  Captain  Preston: 

As  requested,  we  have  reviewed  the  report,  and  the  draft  of  a  new 
Product  Assurance  Program  Standard  and  overall  find  the  report  both  in¬ 
teresting  and  useful.  It  is  not  surprising  that  your  report  recommends 
leaving  MIL-Q-9858A  alone  since  the  same  industry  representatives  worked 
both  on  your  study  and  OODSIA's  review  for  the  DoD  Quality  Assurance 
Council. 

The  DoD  Quality  Assurance  Council  has  recommended  that  no  immediate 
action  be  taken  on  MIL-Q-9858A,  however,  a  working  group  chaired  by  the 
Navy  has  been  charged  with  identifying  problem  areas  that  inpact  on 
quality,  list  them,  rank  them,  and  report  back  to  the  Council.  A  copy 
of  your  draft  report  will  be  forwarded  to  the  Navy  chairman  of  this 
working  group  for  their  use. 

At  this  time,  OUSD's  position  on  MIL-Q-9858A,  is  that  the  docunent 
does  not  need  change  since  it  offers  the  flexibility  needed  for  applica¬ 
tion  within  the  current  and  future  industrial  setting.  The  beauty  of 
the  current  document  is  that  it  describes  the  elements  required  for  a 
quality  program  without  specifically  stating  how  these  requirements  will 
be  docunented.  In  other  words,  9858A  does  fit  the  paperless  environment 
now  existing  in  some  facilities.  Perhaps,  the  recommended  changes  would 
better  fit  into  H50  which  gives  guidance  in  evaluating  a  contractors 
quality  program.  Since  there  is  no  glossary  in  9858A,  namely  because 
MIL-STD-109  is  the  glossary  of  quality  terms,  the  proposed  list  of  terms 
should  be  submitted  to  the  Air  Staff  (proponents  of  MIL-STD-109)  for 
inclusion  in  the  current  revision  now  in  process. 

Defense  standardization  program  policy,  as  well  as  the  rationale  in 
the  previous  paragraph,  negates  a  new  parallel  docunent  to  MIL-Q-9858A 
as  proposed  in  your  report.  However,  if  we  do  decide  to  revise  or 
replace  9858A,  your  proposal  will  be  considered. 


RECEIVED 

FEBli  lUc'-i 

AFWAL/MLTC 
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We  appreciate  the  effort  of  your  team.  While  we  cannot  adopt  your 
rewrites  of  the  documents  at  this  time,  your  efforts  confirm  that  the 
approach  taken  by  DoD  does  reflect  the  latest  thinking  within  both  govern¬ 
ment  and  industry. 


§LU)Q.M-=z=- 

Richard  A.  Stimson,  PhD 
Director 


Industrial  Productivity 
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February  2,  1984 


aewal/mltc 


Richard  R.  Preston,  Capt.,  USAF 
Project  Manager 

Computer  Integrated  Mfg.  Branch 
Department  of  the  Air  Force 
Air  Force  Wright  Aeronautical  Lab.  (AFSC) 
Wright-Patterson  Air  Force  Base,  Ohio  45433 

ATTN:  AFWAL/MLTC  (513-255-6976) 

SUBJECT:  QUALITY  ASSURANCE  MANAGEMENT  STANDARD  REVIEW 


Per  your  request,  I  have  reviewed  the  draft  Product  Assurance  Program 
Standard  and  have  no  changes  to  recommend.  The  proposal  changes  to  support 
the  automated  manufacturing  environment  appear  to  be  very  adequate. 

I  appreciate  the  opportunity  of  reviewing  this  document  and  would  like  to 
receive  a  finalized  copy. 

7/ 


E. 


GENERAL  DYNAMICS 

Fort  Worth  Division 

P.O.  Box  748,  Fort  Worth.  Texas  76101  •  817  732-4811 


Rf-nHVf.D 


31  January  1984 
271-0-34 


FLB  7  1  '.'.'A 

AFWAL/iv'.LVC 


Captain  Richard  R.  Preston 
Project  Manager 

Computer  Integrated  Manufacturing  Branch 
Department  of  The  Air  Force 

Air  Force  Wright  Aeronautical  Laboratories  (AFSC) 

Wright  Patterson  Air  Force  Base,  OH  45433 

Attention:  AFWAL/MLTC 

Subject:  Quality  Assurance  Management  Standard  Review 

Dear  Captain  Preston: 

We  have  reviewed  the  proposed  Product  Assurance  Program  Standard  draft.  The  pro¬ 
posed  changes  provide  the  necessary  flexibility  to  meet  the  needs  of  industry  as 
tlii1  evoluation  of  emerging  technology  improvements  occur  in  design,  manufacturing 
and  quality  assurance  of  products .  The  standard  addresses  both  the  Factory-of-the- 
Future  as  well  as  current  systems. 

Procurements  involving  an  FSD  phase  must  provide  provisions  for  tailoring  MIL-Q- 
9858A  into  a  flexible  program  plan.  The  resulting  plan  and  the  associated  work 
statement  must  form  the  basis  for  cost  estimates  and  pricing.  The  agreed  plan  must 
be  the  overriding  contractual  requirement  with  provisions  for  dynamic  changes  based 
on  day-to-day  results. 

It  should  be  recognized  that  the  current  MIL-Q-9858A  format  is  deeply  imbedded 
by  reference  into  many  government  and  industrial  documents  and  industry  has  resisted 
any  changes  as  reflected  by  the  recent  CODSIA  case  relative  to  a  proposed  "B" 
revision . 

The  urgency  of  incorporating  the  proposed  changes  is  vital  because  of  the  fundamen¬ 
tal  differences  which  exist  in  today's  environment  characterized  by  sophisticated 
technology  changes  being  introduced  into  defense  systems  and  new/improved  manufac¬ 
turing  methods. 

Your  ICAM  committee  is  commended  for  the  objective  assessment  of  MIL-Q-9858A  and 
the  effort  expended  in  proposing  the  necessary  changes  for  supporting  the  automated 
manufacturing  environment. 

We  would  appreciate  receiving  a  finalized  copy  of  the  proposed  document. 


Product  Assurance 


Rr'T.IVED 


Grumman  Aerospace 


Bethpai^e,  New  Vonk  1 1  71-3 

(516)575-2606 
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JOSEPH  P  KINGHELD 
vice  president 


CQS-VP-84-209 
2  February  1984 


Captain  Richard  R.  Preston,  USAF 
Air  Force  Wright  Aeronautical  Laboratories 
Wr ight-Patterson  Air  Force  Base 
Ohio  45433 

Subject;  Quality  Assurance  Management  Standard  Review 
Reference:  (a)  AFWAL/MLTC  letter  dated  19  December  1983 

Dear  Captain  Preston: 

Grumman  Aerospace  Corporation  has  reviewed  the  subject  document  and 
forwards  the  following  comments  for  your  consideration: 

1.  Of  the  three  alternatives  presented  in  paragraph  3.3.2  of  the 
attachment  to  reference  (a)  for  developing  a  new  Product  Assurance 
Program  Standard,  the  one  we  found  most  acceptable  is  the 
MIL-Q-9858A  Rewrite. 

2.  The  Rewrite  accomplishes  the  same  result  as  adding  a  glossary 
to  MIL-Q-9858A,  but  in  a  more  meaningful  and  less  obtrusive 
manner . 

3.  The  proposed  Product  Assurance  Standard,  which  is  still  basically 
MIL-Q-9858A,  adds  a  few  paragraphs  which  will  tend  to  create  more 
confusion  as  to  when,  and  under  what  circumstances,  it  is  to  be 
applied.  It  would  be  more  appropriate  that  the  Software  speci¬ 
fication,  MIL-S-52779,  remain  a  stand-alone  document. 

Only  recently  the  Office  of  the  Under  Secretary  of  Defense  (R&E) 
concurred  with  industry  that  a  rewrite  of  MIL-Q-9858A  was  not  needed  at  this 
time.  Any  proposed  rewrite,  of  MIL-Q-9858A  for  the  purpose  of  providing  for 
the  computer  controlled  "factory  of  the  future"  should  be  coordinated  with 
that  office  to  prevent  any  appearance  of  inconsistency. 

Grumman  Aerospace  Corporation  appreciates  the  opportunity  to  review  the 
Quality  Assurance  Management  Standard  and  if  I  can  be  of  further  service 
please  do  not  hesitate  to  contact  me. 

Sincerely  yours, 

GRUMMAN  AEROSP^E  CORPORATION 

,  Vice  President 
//Corporate  Quality  and 
Safety  Operations 


BOEING  COMMERCIAL  AIRPLANE  COMPANY 


P.O  Box  3707 

Seattle,  Washington  98124 


A  Division  of  The  Boeing  Company 


February  3,  1984 

A-20 20-32-154  RECEIVED 

FEB  1  3  1984 

afwal/mltc 

Richard  R.  Preston,  Capt.,  USAF 
Project  Manager 

Computer  Integrated  Manufacturing  Branch 
AFWAL/MLTC 

Wright-Patterson  Air  Force  Base,  Ohio  45433 
Dear  Captain  Preston: 

We  have  reviewed  the  draft  document  Product  Assurance  Management  Standard  as 
requested  in  your  letter  of  19  December  1983. 

Boeing  Military  Aircraft  Company  -  Wichita  participated  in  the  questionnaire 
and  information  gathering  meeting  during  docunent  development.  Because  of 
that  involvement  we  provided  BMAC  a  draft  copy  and  suggested  they  respond 
directly  to  you. 

The  Boeing  Commercial  Airplane  viewpoint  is  that  the  standard  appears  to  be 
essentially  a  restatement  of  existing  MIL-Q-9858A  in  broader  terms.  The 
standard  does  not  appear  to  offer  many  advantages  for  subcontractors  over 
the  marked-up  version  of  MIL-Q-9858  which  was  recently  reviewed.  Since  we 
already  comply  with  MIL-Q-9858  in  its  present  form  and  the  proposed  changes 
of  the  standard  are  minimal,  we  would  prefer  the  proposed  version  of 
MIL-Q-9858. 

Sincerely, 


F^f^pervisor 
Flexible  Manufacturing, 
Controls  and  Simulation 
Mfg.  Research  &  Development 
A-20 20  931-4310  5H-36 
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Lockheed  Georgia  Conner..  afwaumltc 

A  Division  ol  lOCklH-cd  l.oiiHiialion 
Manell.i  (luoiijia  ;kXHU 


9  February  1984 


Cape.  Richard  R.  Preston,  USAF 
Project  Manager  -  AFWAL/MLTC 
Computer  Integrated  Mfg.  Branch 
Wright-Patterson  AFB,  Ohio  45433 

SUBJ :  QUALITY  ASSURANCE  MANAGEMENT  STANDARD  REVIEW 

Rel'.:  Letter  dated  19  December  1983,  AFWAL/MLTC  to  Joseph  Tulkoff, 
Same  subject 


Dear  Captain  Preston: 

We  have  reviewed  your  draft  Product  Assurance  Program  Standard.  Our 
comments,  which  follow,  note  several  needed  corrections. 

First,  in  section  2.1,  Applicable  Documents,  the  current  standard  used 
for  Calibration  System  Requirements  should  be  MIL-STD-45662.  Second, 
in  Figure  3-1  P.8,  document  4528  should  be  labeled  45208.  Third,  in 
Figure  3-9,  para.  4,  "..  .changes  jof  additions. .  ."  should  read  "...changes 
or  additions. . 

In  general  we  agree  with  the  "overwhelming  response"  that  MIL-Q-9858A 
is  an  excellent  document,  it  serves  its  purpose  well,  and  should  not  be 
changed.  We  also  agree  with  the  first  part  of  summary  statement  4)  on 
Software  Quality  Assurance  (p.27  para.  3.2. 3.3.1);  separate  guidelines 
for  shippable  and  non-shippable  software  should  be  used.  That  is,  M1L- 
STD-52779  should  not  be  applied  to  process  control  software.  However, 
we  do  not  agree  that  MIL-Q-9858/i  should  be  revised  or  replaced  to 
accomplish  this. 

The  changes  noted  in  the  rewrite  of  MIL-Q-9858A  are  not  significant 
enough  to  warrant  the  cost  of  replacing  the  existing  standard.  The 
same  results  can  be  accomplished  through  interpretation  of  the  current 
version  of  MIL-Q-9858A. 

The  Product  Assurance  Program  Standard  does  cover  some  aspects  not 
covered  by  MIL-Q-9858A  and  has  some  good  features,  such  as  the  tailoring 
provision.  However,  it  is  still  very  similar  to  MIL-Q-9858A,  and  it  is 
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questionable  if  its  implementation  would  be  cost  effective,  as  much  of 
what  has  been  added  could  still  be  inferred  from  interpretation  of  MIL- 
Q-9858A. 

Thank  you  for  the  opportunity  to  review  this  document.  If  we  can  be 
of  further  assistance,  please  let  me  know. 


Very  truly  yours. 


L0CKHEED-GE0RC1A  COMPANY 


d  •  Tulkoff,  Director  of 
•  Manufacturing  Technology 


RECEIVED 


LTV  Aerospace  and  Defense  Company 

\  .  •:  A|H  ,  j  ..  v 


FEB  2  1984 

ArVML/MLTC 
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25  January  1984 


To:  USAF,  AFSC,  Wright-Patterson  Air  Force  Base,  Ohio  45433 

Attn:  Captain  Richard  R.  Preston,  Project  Manager,  Computer  Integrated 

Mfg.  Branch 

Subj:  Quality  Assurance  Management  Standard  Review 

Ref:  AFWAL/MLTC  (513-255-6976)  dtd  19  Dec  1983 


1.  In  response  to  your  request,  Vought  Quality  Assurance  has  reviewed  the 
proposed  Product  Assurance  Program  Standard  and  submits  the  following 
observations. 

a.  Our  preference  is  to  leave  MIL-Q-9858A  as  currently  published  with 
a  second  preference  of  adding  the  definitive  phrases  which  will  define  a 
paperless  system  in  lieu  of  the  proposed  Product  Assurance  Program  Standard. 

b.  The  need  for  a  Product  Assurance  Program  Standard  is  not  justified, 
in  our  opinion,  and  runs  counter  to  previous  industry  agreement  that 
specifications  should  not  dictate  "how  to"  but  should  state  requirements. 
Some  of  the  specific  objections  to  the  Product  Assurance  Program  Standard 
are  listed. 


(1)  The  Product  Standard  violates  the  general  opinion  of  industry 
that  "how  to"  wording  should  be  left  out  of  specifications.  There  are  long 
descriptions  of  how  we  will  satisfy  requirements  in  several  paragraphs.  A 
few  examples  are:  3.2,  3.3,  3.4,  3.5,  3.6,  4.4,  and  6.2  the  last  paragraph 
Again,  the  specification  should  state  minimum  requirements  and  let  the 
contractors  decide  how  to  best  comply  in  their  environment. 

(2)  The  Tailoring  Clause  (paragraph  1.6),  while  desirable,  is 
generally  going  to  be  ignored  unless  the  standard  is  issued  in  1,  2,  or  3 
level  requirements.  It  may  even  need  to  have  separate  specification 
numbers.  The  reason  for  this  is  that  most  Procurement  Officers  always 
invoke  the  entire  specification  and  resist  "tailoring." 

2.  In  summary,  we  maintain  that  MIL-Q-9858A  is  acceptable  as  a  workable 
document  in  the  near  term  and  future  factory  environment. 


Respectfully, 


J.  Johnston,  Manager 
QA  Management  Services 
(214)  266-4502 
M/S  49-36 
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At/l/rA&y  P/PPIAA/P  COAt04A/y 
A  Division  ol  The  Boeing  Company  Wichita,  Kansas  67210 

January  26,  1984 
3-4801-6-84-5 


FEB  2  1984 
AFWAL/MLTC 


AFWAL/MLTC  (513-255-6976) 

Attention:  Richard  R.  Preston,  Capt.,  USAF 
Project  Manager 

Computer  Integrated  Mfg.  Branch 


Subject:  Comments  on  Task  D/QA  Draft  Product  Assurance  Program  Standard 


Captain  Preston: 


The  Quality  Assurance  Organization  has  reviewed  the  Project  Priority  1105 
Task  D/QA  Draft  Product  Assurance  Programs  Standard  and  it  is  our  firm 
belief  that  MIL-Q-9858  should  remain  unchanged. 


M.S. 


ger.  Quality  Assurance  Lab  &  Support 
g  Military  Airplane  Company 
K44-36 


JW/KM/ml 
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25  January  1984  coupomatiom 


Ref:  AFWAL/MLTC  Ur  dated  19  Dec  1983 
"Quality  Assurance  Management 
Standard  Review" 

F.1‘E''.'ED 


Mr.  Richard  R.  Preston,  Capt.,  USAF 
Project  Manager 

Computer  Integrated  Mfg  Branch 
Department  of  the  Air  Force 

Air  Force  Wright  Aeronautical  Laboratories  (AFSC) 
Wright-Patterson  Air  Force  Base,  Ohio  45433 


FEB  2  1934 
Ai-T.V.L/K'LTC 


Dear  Captain  Preston: 

McDonnell  Douglas  Quality  Assurance  has  reviewed  the  data  pertaining  to 
"Conceptual  Design  For  Computer-Integrated  Manufacturing  -  Task  D"  as 
transmitted  and  requested  by  your  letter  of  19  December  1983.  We  are 
pleased  to  be  provided  the  opportunity  to  comment  on  the  various  alter¬ 
natives  for  clarifying  existing  Quality  Program  requirements  and/or  for  __ 

establishment  of  new  regulations. 

In  1983,  the  Council  of  Defense  and  Space  Industry  Association  (CODSIA) 
established  Case  9-83.  Basically,  this  CODSIA  Case  focused  on  the  develop¬ 
ment  of  a  questionnaire  which  was  subsequently  distributed  to  Industry 
members  soliciting  information  on  the  current  use  and  status  of  MIL-Q-9858A. 

The  resultant  data  was  evaluated  by  the  CODSIA  Case  Committee  to  determine 
as  well  as  examine  the  Associations’  position  on  the  need  for  a  MIL-Q-9858A 
revision.  This  review  included  any  perceived  need  for  specific  change  to 
MIL-Q-9858A  with  regard  to  the  advent  of  the  automated  factory  environment 
(e.g.,  CAD,  CAM,  Robotics,  etc.).  CODSIA  Case  9-83  has  just  recently  been 
concluded.  The  consensus  was  that  MIL-Q-9858A  is  basically  a  sound  document 
and  revision  to  the  document  would  not  improve  product  quality.  There  was 
also  the  prevailing  position  that  MIL-Q-9858A  clarification  and  interpre¬ 
tation  problems,  which  were  noted  by  a  few  members,  could  best  be  corrected 
by  a  revision  to  Handbook  H50.  As  for  the  "paperless  factory"  or  "factory 
of  the  future",  these  were  considered  at  best  creative  terms  which  describe 
a  different  manufacturing  environment  but  one  which  is  not  defined.  To 
establish  at  this  time  specific  requirements  for  a  still  unknown  method  of 
•operation  is  neither  wise  or  beneficial.  The  adequacy  or  inadequacy  of 
existing  MIL-Q-9858A  requirements  for  future  application  was  considered  by 
most  subjective. 

j 
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We  support  the  study  findings  of  CODSIA  Case  9-83.  However,  it  is  equally 
obvious  that  a  considerable  amount  of  good  constructive  thinking  has  been 
applied  to  your  project  and  its  proposals,  so  it  is  not  suggested  that  they 
be  totally  abandoned.  As  an  alternative,  and  in  keeping  with  the  CODSIA 
Case  conclusions,  it  would  be  more  suited  to  revise  Handbook  H50.  The  re¬ 
vision  could  include  a  glossary  of  words  and  phrases,  similar  to  that  pro¬ 
posed  by  your  project,  along  with  corresponding  clarification  changes  if 
needed  in  the  "Application"  and  "Criteria  for  Evaluation"  sections. 

Sincerely, 


/ 


f  Howard  C.  Todt 

Staff  Vice  President 
Quality  Assurance 
McDonnell  Douglas  Corporation 
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^^Lockheed  Missiles  £  Space  Company,  inc. 

Ill  I  Lock 1  W.1\  Sunnyvak  Caiiioinw  94086 

3  February  1984 


nr::. 

FEE  7 
AFVVAU.  ,LTC 


F.  B.  Allderdice 
0/71-01,  B/103 
1111  Lockheed  Way 
Sunnyvale ,  CA  94086 


Richard  P.  Preston,  Capt.  USAF 
Project  Mgr. 

Computer  Integrated  Mfg.  Branch 

Department  of  the  Air  Force 

Wright-Pat terson  Air  Force  Base,  Ohio  45433 


Dear  Captain  Preston: 

The  Review  Comments  for  the  Quality  Assurance  Management  Standards — 

Draft  and  the  System  Test  Plan,  Task  E  are  enclosed.  Lockheed  Missiles  & 
Space  Company  comments  for  the  Implementation  Plan,  Task  E  will  be 
mailed  to  you  shortly. 

Please  add  Lockheed  Missiles  &  Space  Company  to  the  mailing  list  for  the 
final  version  of  the  Quality  Assurance  Management  Standards  document. 

Very  truly  yours.  * 

F.  B.  Allderdice 
SSD  Manufacturing 

Lockheed  Missiles  &  Space  Company,  Inc. 


/a 

Enc: 
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COMMENTS  FOR  SYSTEM  ENVIRONMENT  DOCUMENT 
TASK  'D' 


In  an  era  of  rapid  and  complex  technology  change ,  the  approach  the  coalition  took 
on  preparing  the  Draft  Product  Assurance  Program  Standard  was  formidable.  The 
coalition  is  to  be  commended  on  its  proposed  new  standard,  and  the  standard  will 
serve  as  a  good  base  for  those  companies  implementing  new  strategies  as  we  approach 
the  factory  of  1995.  As  we  evolve  from  integrated  human  aided  manufacturing  factory 
to  a  integrated  computer  aided  manufacturing  factory,  this  new  standard  will  afford 
us  stability  in  what  we  will  be  doing  to  manufacture  quality  products  as  we  cope  with 
the  dynamically  changing  how  we  will  be  manufacturing  quality  products. 

I CAM  SYSTEM  TEST  PLAN  TASK  ’E’  COMMENTS 

ICAM  System  Test  Plan  Task  'E*  document  is  acceptable  as  written.  However,  it  was 
observed  in  Appendix  A,  that  Fig.  A-ll  and  Fig.  A-16  are  duplicates,  as  well  as 
Fig.  A-9  and  Fig.  A-17. 
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AIRCRAFT  ENGINE  BUSINESS  GROUP  AEVVAL/Mltc 

GENEVA.  ELECTRIC  COMPANY*  '  NEUMANN  WAY  •  CINCINNATI,  OHIO  4S?'S»  So  ■  ?4C  S>ClOC 


February  3,  1984 


Captain  R.R.  Preston 
USAF 

Computer  Integrated  Manufacturing  Branch 
AFWAL/MLTC 

Wright-Patterson  AFB,  Ohio  45433 

SUBJECT:  Review  of  ICAM  Documents  Tasks  B  &  D 

1  know  that  I'm  deliquent  with  these  comments,  but  a  number  of  items  associated 
with  the  start  of  the  new  year  have  interf erred  with  my  response. 

As  I've  said  previously,  we  in  GE  recognize  the  substantial  amount  of  high 
quality  work  represented  by  these  program  documents.  Our  comments  are  intended 
to  provide  Inputs  to  enhance  the  interpretation  and  usefulness  of  the  documents, 
and  should  not  be  interpreted  to  degrade  the  work  effort  represented.  We 
appreciate  the  chance  to  review  these  documents  for  that  purpose. 

Task  B  -  FoF  Conceptual  Framework 

As  I  indicated  early  last  month,  most  of  our  comments  on  this  document  may  not 
be  totally  appropriate  due  to  the  different  nature  of  the  engine  manufacturing 
business  from  the  approach  normally  used  in  the  airframe  industry. 

General  Comments 

Overall,  the  document  provides  a  large  list  that  could  be  used  as  "thought 
joggers."  The  impression  is,  however,  that  many  of  the  elements  and  solutions 
are  repetitive. 

The  Technology  Voids  identified  in  each  section  are  brief  and  not  descriptive. 
The  lack  of  detail  would  present  system  designers  and  vendors,  trying  to  fill 
the  voids,  with  little  from  which  to  work. 

Although  a  business  strategy  is  mentioned  and  discussed,  beginning  on  pg.  3-30, 
more  could  have  been  said;  e.g.  do  we  start  from  existing  islands  of  automation 
in  existing  factories,  or.,  "greenfield"  plants.  What  elements  of  the  business 
strategy  relate  to  the  automation  strategy? 

Cost  justification  of  CAM  is  a  big  problem.  Although  mentioned  on  pg.  3-93,  no 
real  answers  or  suggestions  are  given.  Just  having  financial  data  on  a  data¬ 
base  and  being  able  to  manipulate  it,  will  still  not  answer  the  cost  justifi¬ 
cation  question.  Other  factors  must  be  addressed  here  other  than  the  normally 
considered  financial  considerations. 
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Two  (2)  statements  on  pg.  3-87  and  pg.  3-88  are  contradictory.  One  can’t  strive 
for  "economic  order  quantities"  and  "Just-In-Time"  (JIT)  techniques  simultaneously 

The  term  "automatic  guided  vehicles"  (AGV)  should  be  substituted  for  "robocarriers 
on  pg.  3-89.  AGV  seems  to  be  the  more  widely  accepted  generic  terminology. 

After  reading  the  section  on  Human  Resources,  (Sec.  3-5),  it  is  not  clear  as  to 
where  we  are  going  to  get  the  number  of  skilled  people  U.S.  Industry  will  need. 

The  premise  that  we  can  retrain  non-professionals  to  be  professionals  in  1995 
plants,  (p.  3-68,  first  line)  is  difficult  to  envision.  This  whole  section 
implies  a  large  number  of  people  working  in  1995  factories,  which  may  not  be  the 
case  nor  is  it  the  objective  that  the  automation  and  CAM  people  are  aiming  for. 

In  summary,  a  great  deal  of  the  concept  to  the  1995  factory  seems  to  rest  on  the 
building  of  massive  data  bases,  group  technologies  and  artificial  intelligence, 
ecc.  This  document  covered  superficially  various  system  characteristics  but 
did  not  expand  to  the  degree  we  would  have  liked.  Although  these  are  good 
"motherhood"  type  of  thoughts,  it  would  seem  that  there  are  areas  where  these 
components  of  the  Factory  of  the  Future  cannot  bridge  the  large  gap  between 
today's  world  and  that  of  the  distant  future.  It  is  recommended  that  some 
expansion  be  given  to  filling  the  gap  in  between.  What  will  the  data  base 
provide,  to  whom  and  how?  How  will  artificial  intelligence  benefit  the  FoF  and 
what  are  the  areas  where  it  would  be  inappropriate?  What  are  the  systems 
layouts,  and  priorities  so  future  factory  does  not  become  totally  burdened  in 
dealing  with  these  future  massive  data  bases  and  information  transfers. 


Task  D  -  Product  Assurance  Program  Standard  Specific  Comments 


Page  3  -  Suggest  replacing  MIL-STD-1520A  with  MIL-STD-1520B.  Also, 

MIL-STD-45662  is  not  an  A  revision  that  we  know  of . 

Page  6  -  Definition  of  Quality  Control.  Add  control  of  processes. 

Page  41  -  Para.  4.2.  -  Replace  MIL-C-45662  with  MIL-STD-45662. 

Page  53  -  Para.  2.1.  -  Replace  MIL-C-45662A  with  MIL-STD-45662. 

Page  54  -  Para.  3.3  -  Delete  the  phrase  "providing  an  audit  plan  is 
maintained  by  the  Quality  Assurance  organization." 

Page  55  -  Para.  3.6  -  Paragraph  seems  much  too  detailed  by  defining  "how  to 
information"  rather  than  stating  the  requirement. 

Page  58  -  Para.  4.2  -  Replace  MIL-C-45662A  with  MIL-STD45662A  with  MIL-STD- 
45662. 

Page  59  -  Para.  5.1.1  -  Line  10  -  Delete  the  "s"  from  objectives.  . 

Page  60  -  Line  12  -  change  "inspection"  to  "assurance". 

Page  60  -  Para.  6.1  -  Delete  1st  paragraph. 

Para.  5.1. 


It  is  already  covered  in 
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Page  61  -  Para.  6.2  -  1st  Paragraph  -  Delete  all  after  line  8.  It  is 
covered  in  Para.  3.4. 

Page  63  -  Para.  6.5  -  Line  5,  delete  “and  rework". 

Page  63  -  Para.  6.5  -  Line  7,  change  the  sentence  "The  acceptance  of  non- 

conforming  supplies  into  DoD  inventory  is  a  prerogative  of  an 

•  I 

Page  63  -  Para.  6.7  -  Recognize  use  of  computer  to  maintain  status. 

Page  66  -  Para.  8.1  -  Add  provision  that  when  MIL-Q-9858A  is  applied  to  a 

total  contract,  MIL-1-45208  control  may  be  adequate  for  parts  or 
components  associated  with  the  contract. 

In  addition,  the  following  general  comments  are  provided. 

1.  The  proposed  modifications  to  MIL-Q-9858A  that  allow  for  the  use  of  disc, 
tape,  and  video  display  media  for  the  transmittal  QA  information  is  a  welcome 
addition.  The  additions  proposed  should  be  adequate  for  a  general  specification 
of  this  nature  for  the  foreseeable  future. 

2.  The  proposed  Product  Assurance  Program  Standard  appears  to  cover  all  aspects 
of  a  general  nature  for  preparing  and  maintaining  a  QA  program  without  putting 
unreasonable  restraints  on  a  contractor.  It  allows  for  the  contractor  to 
devise  an  adequate  QA  program  utilizing  the  best  and  most  modern  facilities 
available  to  them.  Any  complaints  in  the  past  of  inadequate  attention  to 
software  QA  appear  to  be  addressed  adequately  to  permit  a  contractor  to  devise 

a  software  plan.  There  is  a  reference  to  MIL-S-52779A,  Software  Quality 
Assurance  Program  Requirements,  that  we  do  not  have  ready  access  to,  that  probably 
should  also  be  reviewed.  This  specification  may  also  need  updating  to  make  it 
current  with  the  present  state-of-the-art  in  software  design  and  documentation 
methodology . 

Please  contact  me  if  you  have  any  further  questions  regarding  these  comments. 


Very  truly  yours, 

GENERAL  ELECTRIC  COMPANY 


E.  E.  Welsmantel,  Manager 
Manufacturing  Technology  Programs 
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•uujtn  Quality  Assurance  Management  Standard  Review  (Your  Ltr,  19  Dec  83) 


to  AFWAL/MLTC 

1.  The  technical  report,  ICAM  Conceptual  Design  for  Computer- Integrated 
Manufacturing,  Task  "12“  Quality  Assurance/Quality  Control/Technical 
Requirements/Tasks,  have  been  reviewed  and  the  following  comments  are 
provided: 

a.  The  Government/ Industry  survey  appeared  as  the  most  Interesting  part 
of  this  technical  report.  Many  of  the  comments  we  can  agree  with  as  stated. 

(1)  The  first  section  of  the  survey  results  are  about  what  we  would 
expect  from  contractors  under  our  cognizance. 

(2)  The  Quality  Assurance  questionnaire  of  seven  questions  for  the 
present  and  factory-of-the-future  environment  are  addressed  separately  as 
follows: 


£.  Current  application  of  MIL-Q-9858A:  We  agree  with  all 
comnents  except  the  last  comment  "Doesn't  need  changing".  We  are  of  the 
opinion  that  MIL-Q-9858A  could  use  some  refinement  for  today  -  as  well  as  for 
future  application.  Some  of  which  have  been  touched  upon  In  this  report. 

J).  Problems  of  compliance  with  specifications/ standards:  We 
don't  totally  agree  with  the  results  of  this  question.  Many  times  the 
referenced  specifications/standards  are  used  as  an  excuse  not  to  accomplish 
good  sound  management  practices,  also  they  are  used  as  excuses  to  increase 
costs,  even  when  contractors  have  a  system  In  full  compliance  with  the 
specifications/ standards. 

£.  Impediments/opportunities  for  the  factory  of  the  future: 

Most  of  the  comments  listed  can  be  accomplished  as  they  occur  within  the  scope 
of  the  existing  specifications/standards.  Two  of  these  statements  are  of 
interest  when  tied  together:  "Prepare  for  paperless  documentation"  and 
"provide  for  product  traceability  and  Identification  of  acceptance  status 
while  in  process".  They  could  be  a  real  cost  savings  to  the  government  as 
well  as  the  contractor.  However,  somewhere  documentation  printouts, 
mltrofish,  etc.,  must  be  accomplished. 

_d.  Perceived  necessary  changes  to  specifications/standards:  The 
additional  thoughts  don't  appear  to  have  any  significant  Impact  to  warrant 
changes  to  present  documents  for  an  automated  factory. 


ji.  Reliability/maintainability  feedback:  While  this  is  a  big 
"Q"  total  MIL-Q-9858A  multifunctional  Quality  Program  specification ,  It 
doesn't  mean  that  it  is  a  quality  functional  problem.  This  may  need  further 
attention  in  the  typical  APPRO  engineering  documents. 

_f.  Impact  of  paperless  factory:  The  so  called  paperless  factory 
could  be  implemented  using  the  existing  Quality  Program/ system  specifications/ 
standards  without  changes.  The  specialty  specifications/standards  that 
contain  quality  requirements  will  require  changes  as  the  paperless  factory 
becomes  a  reality.  Acceptance  of  electronic  media  for  data  and  part  status 
Including  support  equipment  maintenance/calibration  has  been  utilized  under 
control  of  existing  specifications/standards.  Tailoring  of  specifications/ 
standards  has  proven  over  the  years,  for  the  most  part,  a  beneficial  technique 
for  various  phases  of  a  weapon  system  acquisition  contract.  Inspection  stamps 
as  we  know  them  may  disappear.  However,  we  expect  to  see  some  type  of  a  code 
system  that  would  replace  stamps. 

jj.  Development  of  a  new  Product  Quality  Assurance  Standard: 

These  comments  appear  to  be  appropriate;  "make  no  change"  to  the  existing 
Quality  Assurance  specifications/ standards.  The  seven  indicated  statements 
are  presently  in  effect  or  under  evaluation  by  the  appropriate  Air  Force 
Office  for  further  implementation. 

(3)  Quality  Assurance  Meeting  Summaries: 

j*.  The  present:  Basically  our  comments  above  address  this  part 
of  the  report.  An  approach  to  the  field  data  reporting  problems  could  be  to 
contract  with  the  private  sector  to  perform  an  evaluation  on  the  existing 
approach  and  provide  recommendation  on  how  to  best  obtain  this  needed  data. 

b.  The  future:  We  see  evolution  as  the  approach  to  the  future 
factory.  AgaTn,  I  want  to  emphasize  that  we  are  not  against  constructive 
changes  to  the  existing  Quality  Assurance  specifications/standards.  Emphasis 
must  be  placed  on  establishing  terms/definitions  for  the  plant-of-the-future. 
Assure  training  is  conducted  relating  the  new  terms/ definitions  to  existing 
Quality  Assurance  specifications/standards  for  the  future  factories.  This 
would  Include  the  more  advanced  self-adjusting  automated  equipment. 

(4)  Conclusions  (page  30): 

a.  Our  conclusions  are  quite  different  than  those  10  listed. 
While  each  of~ the  10  conclusions  have  certain  Impact  on  contractor-government 
cost  they  still  have  some  merit.  However,  even  they  alone  are  not 
justification  to  generate  new  or  modify  the  major  existing  Quality  Assurance 
Program/ Systems  specification  standards. 

b.  Our  conclusions  and/or  recommendations  are  as  follows: 

(1)  Stop  and  reassess  your  overall  approach  to  this  task. 

(2)  MIL-Q-9858A  can  be  applied  successfully  in  the  automated 
environment  for  factories  of  the  future. 


(3)  Recommend  updating  some  of  the  specialized  specification/ 
standards  such  as  MIL-STD-109,  Quality  Assurance  Terms  and  Definitions. 

(4)  Recommend  updating  DOD  Industry/ government  wide  accepted 
Handbooks  for  Evaluation  of  Contractors  Quality  Program/Systems,  such  as  H-50, 
H-51,  H-52,  etc.  This  is  where  the  payoff  could  really  take  place.  It  would 
allow  for  the  future  plant  layout  diagrams,  terms,  options  on  how  to 
accomplish  a  requirement. 

2.  Based  on  our  review  and  evaluation  of  this  technical  report  leading  up  to 
page  31,  para  3.3,  Product  Assurance  Program  Standard,  I  see  no  reason  to 
attempt  an  exhaustive  review  and  evaluation  of  the  proposed  rewrite  of 
M1L-Q-9858A  or  the  draft  Product  Assurance  Program  Standard. 

3.  Please  keep  me  informed  of  your  progress  on  this  difficult  task.  Let  me 
know  if  I  can  be  of  further  assistance  on  this  project.  Thank  you  for 
including  us  in  your  review. 


DAVID  L.  JSNES/  Chief 
Performance^aftd  Evaluation  Division 
Directorate  of  Quality  Assurance 
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replyto  fine  (G.  Mays,  57371)  29  May  1984 

ATTN  OF:  J  J 

SUBJECT  ICAM  CDCIM  Task  "D" 


io.  AFWAL/MLTC  (Capt  R.  Preston) 

Overall  this  document  is  very  good,  I  do  have  two  comments  that  are  "nits" 

1.  Inputs  from  Manufacturing  and  Design  Engineering  need  to  be 
considered. 

2.  Contractor  should  be  required  to  use  "Quality  Cost  Data"  as  a 
management  element  of  the  production  program  -  not  just  the  quality  program 

GILBERT  A.  HAYS 
Manufacturing  Engineer 
HQ  AFCMD/PDP  Collocated 
Computer  Integrated  Mfg  Branch 
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APPENDIX  D 

AN  APPROACH  TO  REQUIREMENTS  DEFINTION  ENHANCEMENTS 


The  following  Appendix  outlines  an  approach  that  might  be  used 
to  enhance  the  definition  of  requirements  for  a  Quality 
Assurance  Program  Management  Standard.  The  recommended  approach 
is  to  create  a  "To-Be"  model  of  Quality  Assurance  in  the 
automated  environment  of  the  future  and  use  this  model  to  verify 
that  the  "future"  Quality  Assurance  Program  Management  Standard 
effectively  addressed  the  perceived  requirements  of  the  ''future" 
automated  factory. 
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I 

It  is  a  truism  that  in  the  future  factory  how  things  will  be 
lone  will  be  different,  to  various  degrees,  from  how  they  are 
being  done  today.  In  IDEFo  function  modeling,  the  how  can  be 
considered  analogous  to  a  mechanism.  Furthermore,  standards  and 
j  specifications  are  imposed  (again  to  various  degrees)  on  an 

activity  in  order  to  establish  a  measure  of  conformity,  thereby 
avoiding  chaos  and  confusion  in  the  execution  of  that  activity. 

In  IDEFo  modeling  standards  and  specification  can  be  analogous 
to  the  con trols  which  limit  or  constrain  the  activity.  If  one 
assumes  that  the  inputs  and  outputs  of  a  factory  of  the  future 
'<  will  remain  relatively  constant  vis-a-vis  today's  factory  (in 

1  the  broad  sense  -  raw  materials  will  still  be  inputs,  finished 

goods  will  still  be  outputs,  for  example)  then  an  examination 
of  tomorrow's  anticipated  mechan isms  can  be  used  as  a  guide  to 
defining  tomorrow's  anticipated  con  trols .  Another  way  of 
expressing  the  same  thought  is  as  follows: 

"Today's"  standards  and  specifications  (MIL-Q-9858A,  e.g.)  were 
"driven  by"  the  need  to  control  "today's"  methods,  equipment, 
paperwork,  etc.  ( mechan isms ) .  The  requirements  for 
"tomorrow's"  standards  and  specifications  (new  Quality  Assurance 
Program  Management  Standard,  e.g.)  will  be  driven  by  the  need  to 
1  control  "tomorrow's"  automated  methods,  automated  equipment, 

'  computerized  data  systems,  etc.  Pictorially,  that  is 

illustrated  in  Figure  D-l. 

Figure  D-2  shows  that  with  the  development  of  a  "To-Be"  model 
for  QA/QC  the  requirements  for  a  proposed  new  Quality  Assurance 
|  Program  Management  Standard  could  be  examined  against  the 

backdrop  of  the  probable  future  tools  (mechanisms)  of  the 
automated  factory  and  the  results  of  this  examination  could  be 
used  to  modify  the  Quality  Assurance  Program  Management  Standard 
where  appropriate. 

It  is  recommended  that  an  IDEFo  "To-Be"  model  of  Quality 
Assurance  be  developed  and  used  to  further  enhance  the  Quality 
Assurance  Program  Management  Standard  presented  in  this  document. 
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